Sluggish library browsing with huge library

THis has been something I have noticed in the Engine Prime software. Creating and modifying/sorting crates takes an exorbitant amount of time.

3 Likes

I suspect it’s the library size, not the number of tracks analyzed.

1 Like

I suggested something related before that would be ideal:

  • Automatic background search database build (when plugging in unanalyzed card/drives tags will be read in the background, quickly make the search function available to all tracks. Further analyzation could be done once the track is loaded)

I wonder is this is something Denon is aware of and sees as an issue, or is it just me, @paul_denondj ?

With the Prime4 having a bay for a drive I guess users will expect to truly use the space of their 1TB drives with proper navigation, searching and loading times.

Thanks

The library size you mention is well beyond our suggested library size recommendations. We set these to help avoid instances like this. Please see below:

Someone has suggestions on how to improve the library in another thread

Bye bye laptop? Am I ready?

Just to be clear, that was done using the PC version of the database - I’ll check the USB version of the database probably later today but I don’t expect to see much difference.

Also @paul_denondj I really don’t think that’s a good enough answer - clearly there are optimisations the developers can do on the database side. For example, a project I was working on recently was running a version of Windows CE and had over 50,000 users on the SQLite database - I don’t know the exact hardware but it was nowhere near as powerful as the SC5000 hardware. Doing a search for username (which searched first name, last name, email address) took 2.81 seconds on average. The main reason for this was good indexing and views.

You can also structure queries to only bring back the information in each table that you actually use, rather than all of the fields - this especially helps if the fields contain lots of long strings or other long datatypes (images, blobs etc).

The SC5000’s taking 7-8 seconds to search through 50,000 records with the hardware available is therefore pretty unacceptable in my opinion.

Additionally, you guys are recommending a 1TB SSD for the Prime 4 - do you really think people are going to limit their library to 10,000 tracks on a 1TB SSD? That’s just not realistic.

2 Likes

Fair comment and good info, thanks Jonny

I’ll ask our software Dev team expert to chime in with more insight…

Best

Paul

1 Like

Even with extended mix wav files, you’ll easily get over 50000 tracks on 1TB

Well, there are conscious people and there are stubborn people. Whether they put a limit to themselves or not it is their choice. What matters is the fact that every piece of equipment has different specifications and limitations and they should be used accordingly. What we want is not always what we can obtain from a given equipment.

2 Likes

It’s the same library database. I run Engine Prime on a 2TB USB drive with 80,000 tracks on it that I then use on the players direct. I don’t prep little thumb drives with a single set on it or something. If I wanted to stick to 10,000 tracks on a drive I’d just be using the Hanpin players with that as their hard cap. I use the same drive on the Pioneers and an otherwise-identical NTFS read-only copy of the drive on the Geminis.

I think it almost certainly is the same database (aside from different file paths etc) but I just want to be sure. It is entirely conceivable that they might be doing further optimizations on exported databases.

For a PC/Mac database optimization doesn’t matter too much as there often won’t be much benefit due to the raw computing power available (there are of course exceptions to this depending on the query itself - a lot of report queries need optimizing for instance as they can cross-reference many tables and make the query a lot slower, or there could be millions of rows returned over time for reports etc). But once you step into lower powered hardware, even simple queries benefit from optimization.

1 Like

This makes me happy. Knowledge dropped, listened to and acknowledged.

The Hd2500 is about 12 years old and had instant search results from a database that could handle 50k tracks per usb drive - and could accept 4 external drives and an internal drive also with 50k tracks. That’s 250k tracks all on instant return results.

You couldn’t search all drives at once though and you didn’t have any way of linking a HD2500 to another HD2500 so no networking either

But instant searching on huge databases 12 years ago.

Gotta be do able now denon

Thanks for answering @paul_denondj. I am aware of that, and it’s reflected in my initial post, where I start by noting that I am aware of that recommendation. Indeed the paragraph you quote is mine, copy pasted from the Denon website.

That said, I’m just calling out the limitations of it, and that that’s limitations could be solved by a better designed database and it’s not a necessary hardware limitation.

Indeed, that limitation will frustrate many Prime4 users, as the Prime4 is sold with a drive bay and Denon recoments 1TB SSD drives, which are capable of storing much more than 10k songs. To a certain extend and trying not to be harsh, I would say that advertising a product with a drive bay and with so many USB ports and is a bit misleading once you discover such limitation.

Hope that makes sense, Thanks.

1 Like

Hi All

Just revisiting this topic after getting some good news from our Dev/Product team. So…

It seems that the previously mentioned ‘recommended limit’ of 10,000 tracks was set some time back and had carried on into recent messaging for Prime 4. And this is where the voice of the Forum members here carries such weight, for which we’re grateful for your input. The team have revisited this and stress tested the system again recently to find a more appropriate track limitation recommendation, especially when using SSD drives.

I’m very happy to report that Denon DJ now officially (have tested up to and) recommend a song capacity (differing this time to a ‘limitation’) of 50,000 tracks. The caveat however is that this is based on the strong recommendation to use SSD drives to access those 50,000 tracks!

So that’s the official place we are at, which I feel is awesome news and hopefully a great sigh of relief for all members here and end-users. Plus, this shows how YOUR voice is listened to, action taken, and results achieved!

To add, some of our Denon DJ team have reported usages in the field again (and we can’t stress this enough) on SSD, that they have had no problems or issues accessing libraries containing 70,000 tracks, reporting a good experience on search, load and playback.

Best

Paul

*Edit: The figures above relate to the Prime 4 (being a ‘closed system’)… I’m awaiting more info on the ‘network system’ (SC5000 etc) incoming from the Product team and will chime back into this thread soon…

7 Likes

These tests were carried out with every supported format, including uncompressed (.wav) files? 50000 tracks averaging 8 minute each would require a HDD of about 2TB.

Thanks for the response @paul_denondj.

Are these new figures already working for current firmware in the SC5000s?

If so, unfortunately, I’m still experiencing the issues described in my first post - I have about 50k tracks (flac and mp3) in one SSD Samsung 1TB drive connected to the back USB 3.0 port, and every operation I perform (search, change albums) take 8 seconds.

reporting a good experience on search, load and playback

Can you confirm that the 8 seconds I’m experiencing falls outside the mentioned "good experience " and that they should not happen? Do you think that what I’m seeing in my units is a technical problem and they may be faulty?

Also, it would be useful to get some info from the dev/product team if the number of audio tags in the tracks matters or are irrelevant (I automatically tag my tracks with information from Discogs including year, record label, catalog number, country and release format).

Thank you.

I think he said the figures were on a “closed” system, rather than a networked system, but he said he was going to check.

Rather than only looking at SSD it is also worth looking at HDD with large cache also

I use SSD’s in everything (exported music drive, OS drive, storage drives). I do not run into issues on the players. My issues are in Denon Engine Prime management software. After finally converting my entire collection over, the software is really tough to use. My home computer is running an Intel 6950X, 1080TI, 64GB of RAM, so I know it isn’t a hardware issue.