Substantial improvements to Library Search performance (the Updating thing) / Faster search larger libraries

  • What is the feature or ability you would like to have? Substantial improvements to Library Search performance. Search results returned instantly / within a few seconds even for libraries with tens of thousands of tracks.

  • How will this feature help you and others? The unnecessary waiting for a search result is avoided. The workflow is faster for every user. Party DJs can quickly find and play short parts of tracks.

  • Is this feature available in an existing product? If so, what product? Numark D2 Director, Numark DDS, Denon HD2500, and more

  • Does a workaround currently exist? Not for Prime

  • How often would you use this feature? All the time, especially on certain concepts where playing short pieces of tracks it is necessary to be able to search very quickly

  • Is there any additional information you’d like to add? The current search speed is stopping some of our DJ’s to change over to the Prime series unfortunately. Certain units (sometimes more than 10 years old) search faster than the Prime 4.

9 Likes

My P4 search isn’t that slow however my back up Numark iDJ2 brings track up instantly

1 Like

+1. I’d love the two SC5000s I have, but this is really bothering me. To me, this is by far the biggest issue of the system, and with my specific workflow and usage of the decks much worse than the reported issues about BPM detection.

Why not just be a little more Proactive on the DJ’s Part. While yes it’s nice to have your entire collection with you on a Stand Alone unit, SC-5000 and or Prime 4. I keep my most popular tracks on a simple 64 Gig thumb drive, and use that as my primary source. Maybe a second Thumb Drive for possibilities options. Searching the drive is a lot quicker than constantly referring to and searching your Entire Drive. I search that drive maybe 2-3 times in a Gig.

I would please suggest users that don’t see this as an issue simply not to like the original post. Some of us see this as an issue. I have been reporting this repeatedly during months. Now that the market has spread with the Prime4 I’m starting to see other users to share my concerns.

We do know all know that:

  • SSD drives make the issue less.
  • DJs are not likely to play 100k songs one night.
  • Mastering and knowing your collection and trimming your library helps.
  • Working with crates helps.
  • Splitting your collection in several drives helps.
  • Jeff Mills and Grand Master Flash used vinyls and they survived. There is no real need to huge libraries enhanced with fast search.

We do get it.

Some of us however think that is a problem that should not be there, that the system has more than enough processing power to get this right, and that Traktor, Serato and other hardware got this right 10 years ago.

Also, as a software engineer myself, I suspect (I see symptoms although I can not demonstrate, it’s not my job to do so, and no one has asked me to do so) that the design of the library database may be culprit of not only this issue (for the ones like myself see it as an issue) but other issues as well, such as the excruciatingly slow performance when Engine imports new songs into big libraries or simply loads when launched.

We are all aware this is not a blocker when using the decks and that Derrick May could still perform OK using these as in current state. We know that there are workarounds on our side, but I personally like to try getting Denon’s attention on it eventually.

They will ultimately decide whether to put more resources on this or is OK as it is - (and the prospective buyers and industry will also behave accordingly).

2 Likes

please make the search much faster getting it near the speeds I get using Virtual DJ with my MCX8000 and my earlier Numark ndx500’s and Mixdeck Quad

The §engineers, or maybe that should be “Engine”eers that write the database code and firmware etc should take a look at the pre-inMusic units. The DN-HD2500 search wasn’t just damn quick it was INSTANT even when you searched by two criteria.

In its defence, the on-machine search on Primes, where you can type two words separated by a space, eg: “Flo Low” gets you Flo rider: Low… and it’ll home in on only tracks which meet BOTH words, is excellent.

I agree with the other idea that someone mentioned that THE database should actually be saved twice on the relevant drive so that in the situation of a corrupt database, the user is told “Main Database is corrupt. Did you want to load the backup database and copy the backup database to a new main database file behind the scenes?” rather than “database is corrupt - format drive”

3 Likes

My question here is, what causes the updating delay. Typically what the preparation software (EP in this case) does, is create an index(ed database). The thing about an index is that it allows for lightning fast searching in big volume data.

So, even with less than major computing power, index searching should be quick.

1 Like

This was an issue before the last firmware update but sort of a non-issue now, since other users reported faster searching since the last upgrade.

How fast or slow everyone’s drive is.

I agree though that denon shouldn’t be writing bloatware and expecting everyone to go for hyper fast SSD when denons 10 year older machines were able to beat Primes in a search results race. It’s just sloppy programming today.

The database is indexed. I think, however, is not the best design.

Metadata fields such as album title, record label, etc, don’t have a column in the songs table. Instead, they have another table able to store any kind of values. This table has a row determining which kind of metadata the value is.

With this design the database is easy to upgrade (if they introduce a new type of metadata they don’t need to alter the table, just create a new type determining which kind of metadata the value is and insert new rows in the metadata table).

It may make queries slower because:

  • metadata values are not in the table of the tracks. A joint is needed.
  • as they don’t know upfront the kind of metadata of each row in the metadata table, they can’t limit the amount of data queried using one SQL projection. They need to retrieve all the metadata values for each track (even if they are not being used) and filter later.

I think Engine may not be the best programming in the world. Although no one has seen the code, looking at how it works raises some red flags to me. Ie: constant re-read from the disk when performing operations that should not have triggered them, unacceptable (and clearly unneeded) time when doing some operations that seem to be doing unneeded work…

Navigating the library in the player also exposes some behaviour that look like a red flag to me. Ie: sometimes, when I go back from search results, I arrive to the place I was when starting the search. Other times it takes me back to the root of the library. I obviously cannot explain this, but this non-deterministic behaviour is a sign of defensive programming, race conditions and patching legacy stuff without unit tests.

My two cents - apologies if I sound pedantic or overly opinionated. Feel free to delete this post. I just feel frustrated.

I do not entirely agree with the fact that it is no longer an issue. I think it can be a lot faster. It has become somewhat faster since the last update, and I am very happy that attention is being paid to it, but there is still so much to gain in search speed.

For me personally, searching for speed on a Sata drive is currently the biggest annoyance of the Prime 4. And yes I am aware that it can be faster with SSD, but not all of our DJ’s want or can do that. If a Numark D2 Director from 2007 can return search within a second on SATA then I think the 2019 Prime 4 with all it’s horsepower should also be able to do that trick. It is a bit frustrating when you want to switch a track at the last minute but cannot get the number up in time. Or when you are staring at your screen for 10 seconds after typing a customers request to be able to answer whether the track is present or not

Perhaps the common link between faster older decks like the d2 and the HD2500 with their instant search results, compared to today’s Prime SC5000 slightly slower search results speed is: networking.

On the sc5000 people expect to only keep one disc drive in a good state of housekeeping and to share that drive over a network of 2 or more players. The database files must have to be structured in several small files to allow for their sharing and access across the network without slowing down or hogging the data traffic flow between decks and mixer. Like I wouldn’t want my pitch sync or bpm effects to suffer as I slowly increase track pitch on one bpm sync’d deck I I happen to be searching and filtering results at the time.

The old d2 and hd2500 didn’t network so didn’t need to think about network agility.