(Mostly) complete database of every classic trance track (100.000+) [status: 50%]

Joined
Aug 23, 2022
Messages
71
Thanked
67
DATABASE LINK

STATUS: 50% (1988-1997, partially filtered/cleaned)
ACCURACY UPON COMPLETION: 90-95%



Why I created this database

I'm writing a book about classic trance music for a while now. You have probably read my post about it on the forums some months ago, but the project's scope and goals were significantly changed (and grown) since then. The database you can see here is a direct result of this process. My original intent with it was to aid my research for the book and make processing large amounts of data (and also the writing) faster. But then I quickly realized that it could be a useful and valuable source of information for all trance lovers out there, hence, I decided to make it available to everyone.


About the database

The goal is to compile all trance (and proto-trance) tracks made between 1988 and 2006 and make them available in a single database, along with basic details such as artist names (obviously), the lengths of the tracks, and links to their YouTube uploads. Discogs.com is being used for the most accurate and complete information possible. Once finished, the database will have almost every classic trance track ever made listed in it. I say 'almost' because I had to make some minor compromises to save a significant amount of work time.


The compromises
  • First of all, the database won't feature any track below 4 minutes. Trance tracks generally tend to be around 6:30 long and even the shortest classics I know are around 4:30 in length. I think 4 minutes is a fair compromise - most tracks under that are just radio/album/single/7" versions anyway or tracks mislabeled as trance on Discogs. This way, I can filter out at least 5-10 thousand (probably) unnecessary tracks that I don't need to listen to.​
  • Second, to make the database, I only used vinyl releases as a reference, so no data was grabbed from CD, cassette, or digital audio tape releases. I chose vinyl releases because according to my research, in the classic era, pretty much everything came out on vinyl, from the well-known anthems even to the most obscure tracks that no one has ever heard of. Getting tracklists from CDs etc. would have resulted in significantly more data to process, but in the end, it would have resulted in tons of additional duplicates and barely any additional, actually new tracks (not to mention tons of other tracks from other genres, as CDs tend to have a lot of mixed style compilation albums).​
  • As part of the manual removal process of duplicate tracks, due to human error/mistakes, it will more than likely happen that some tracks that are in fact, not duplicates, will accidentally get deleted. This will probably only result in the loss of a few hundred tracks at max (out of the 100.000), still, I wanted to clarify this.​

It's a guesstimate, but you can expect this database to have at least 90-95% of all classic trance tracks ever produced once it gets completed.

1685738504596.png


IMPORTANT
The Style column doesn't necessarily show the style/substyle of the given track correctly, as this data was grabbed from the given vinyl release. Thus, it primarily represents the style of the album itself, not the individual tracks on it. So don't read too much into it. I only left this column in the database for my own work purposes, but I might remove it later.


Database status

The database currently lists all tracks from between 1988 and 1997. Duplicate releases were (mostly) removed. Radio/single/7" edits/versions/mixes and acapellas will also be removed later. Once I have all tracks from all years, further duplicate releases (cross-year releases) will also be removed, and all full releases from all years will be presented in a single master list.

There are currently around 50.000 tracks in the database and it is roughly 50% complete.


How the database is being created
  1. Using SyncApi (Google Sheets add-on), the following information was grabbed for each Trance vinyl release between 1988 and 2006 from Discogs.com: release name (artist name + album name), year of release, style, and API URL.​
  2. Using Microsoft Excel's Power Query, further data was grabbed from the previously mentioned API URLs, including each release's entire tracklist, along with corrected artist names.​
  3. Duplicate tracks were removed with a combination of an automatic and manual process. Any track that can be seen on the list always shows its very first release date, along with its longest possible length.​
  4. Radio/Single/7" Mixes/Edits/Versions were removed because these are always cut-down versions of the full, uncut versions of pieces of music that are already on the list. Acapellas were also removed.​
  5. Hyperlinks were added to all tracks, making it possible to quickly start listening to them on YouTube.​
Creating the database is mostly a manual process, as a lot of things can't be automated due to a communication problem between Excel and Discogs' server and API client, so it takes a lot of time. Points #3 and #4 are not complete or fully complete as I'm still currently making the database.


Extra: tier list

I'm also creating a different database that will grow over time, but I'm not yet ready to make this one open to everyone yet. This will be used to compile every single worthwhile trance track (4.0+ stars, with a few exceptions where notes are included). Album names and artist names will redirect users to their corresponding Discogs page. Clicking on a track name will open a link to its most popular upload on YouTube. Release year and track length are color-coded to show in which trance era the given track came out and in which length category it falls. Clicking on a year switches to its corresponding tab in the spreadsheet (e.g., if you click on 1993, you will see a separate tab where you’ll be able to see all trance releases from that year). The ratings are based on my personal opinion and experience. The Web Purchase column shows the following options: Beatport, Juno Download, Apple Music, and Amazon Music (with hyperlinks if the given track is available to buy somewhere). Sometimes notes are included (e.g., if the track is available to buy but only in a shorter/mixed-over form). CD/WEB shows whether the track has seen a digital release or if it's a vinyl-only release. I may share the spreadsheet sometime in the future.

1685738023694.png
 
Last edited:

Nerio

Member
Joined
Aug 2, 2020
Messages
130
Thanked
82
YouTube channel Trance-M with its 5000+ trance collection has some damn good competition.
Are you also using ChatGpt for help in this project?
Some of the songs are really obnoxious.
The idea of rating songs by personal preference is a good one, my idea would be if everyone could rate all the songs they listen to from that list and then compare the ratings. It would be easy to see who likes what.
But some are great for example 4Voice - Fairlight 1 for the year that it's released I like it.

But this is exactly the kind of project I've been waiting for, because neither Spotify nor other streaming services have all such "hidden gems" maximum YouTube. And here you put them even more nicely in the list for us, great, I'm going to listen.
 
Last edited:
Joined
Aug 23, 2022
Messages
71
Thanked
67
Are you also using ChatGpt for help in this project?

I'm not using ChatGPT for creating and filling up the database, but I'm using it for other things, like collecting trance music-related historical information, etc.

Some of the songs are really obnoxious.
A lot of tracks tagged as trance, hard trance, etc. on Discogs pre-1993 (proto-trance era) have nothing to do with trance in the majority of cases, in my opinion (or they are partially trance at best). Still, first I wanted to create a full master list including every track that Discogs categorizes as a form of trance, then start making my own list (focused on the best tracks) as I progress with the book.

The idea of rating songs by personal preference is a good one, my idea would be if everyone could rate all the songs they listen to from that list and then compare the ratings. It would be easy to see who likes what.

Yeah, I actually want to enable Google users to vote on the tracks, but only after I created a more streamlined/refined list.

There are 100.000 trance classics?
More like 100.000 tracks that are categorized as a form of trance music that came out in the classic era. So they are not necessarily classic in the sense that they are so good and timeless, they are classic in the sense that they came out in the classic era and (more than likely) have that classic sound/style.
 
  • Thanks
Reactions: JohnnyBarefoot1992
Joined
Jul 19, 2022
Messages
145
Thanked
228
This is awesome! Thank you for the hard work. This will make finding breaksy trance a lot easier and I am excited to dig :).

Edit:
One request: would it be possible to add the Label in a column? Would be really interesting for discovery to see which labels were consistently releasing cranking trance tunes. We all prob know the big ones, but it would be very valuable for the way I collect music anyway. Cheers thanks.
 
Last edited:
Joined
Aug 23, 2022
Messages
71
Thanked
67
One request: would it be possible to add the Label in a column?
Unfortunately, I don't have the API links for the individual tracks from between 1988 and 1997, because I deleted them (API contains data like labels, etc.). I assumed what's important to have in the database are track name, artist, length, and maybe style (plus I have to keep the database's size in mind so I won't run into Google Sheets' limits). But I might add label data to the other, much smaller database (tier list) that will only contain the best trance tracks.
 
Joined
Jul 19, 2022
Messages
145
Thanked
228
Unfortunately, I don't have the API links for the individual tracks from between 1988 and 1997, because I deleted them (API contains data like labels, etc.). I assumed what's important to have in the database are track name, artist, length, and maybe style (plus I have to keep the database's size in mind so I won't run into Google Sheets' limits). But I might add label data to the other, much smaller database (tier list) that will only contain the best trance tracks.
Would be shame to lose some of the data due to google sheets' limits, just sayin. Most of us will download the database anyway and stick it in Excel or Python. Label is really really important to most collectors I know anyway. I can likely reconstruct that column using the scraped Discogs Access DB and some search results, but recommend that you keep larger versions of the database with the API links (if possible). Many thanks again!
 
Joined
Aug 23, 2022
Messages
71
Thanked
67
Would be shame to lose some of the data due to google sheets' limits, just sayin. Most of us will download the database anyway and stick it in Excel or Python. Label is really really important to most collectors I know anyway. I can likely reconstruct that column using the scraped Discogs Access DB and some search results, but recommend that you keep larger versions of the database with the API links (if possible). Many thanks again!

I have a different database that has all the API links, but it only shows their corresponding trance album, not the tracks on them. I could make that available to the public.