Fat, 4gb file size limitation

I accidentally posted this in the wrong part of the forum. Hopefully it’ll work now… So I have an issue here that seriously needs attention. I presume that this issue would mainly affect the Mcx8000 users as the newer Prime gear can access different format memory devices. But I wouldn’t be surprised if some Prime users also experience this purely due to not knowing about it. Anyway… I recently became aware that the FAT file system has an individual file size limit of 4gb. This is a big problem for dj’s with large music collections. EP creates a p.db file that grows in size according to the size of your collection. When the p.db file hits the 4gb limit, one of two things happen. 1) Engine says ‘your database is corrupt’, or 2) It simply refuses to analyse anymore tracks. #2 I could almost live with, if it at least added my music to the database without analysing it, unfortunately, EP refuses to add any un-analysed files to the database. This leaves us with large amount of music that we aren’t able to search through. So we really need an alternative here. Either: 1) Somehow reduce the amount of data stored in this file (not ideal) or 2) Create a 2nd p.db file that EP, Prime series hardware & the 8000 can access. Perhaps there are other ways of addressing this, but this is just ideas off the top of my head. I really hope Denon can come up with a prompt solution for this issue.

1 Like

This is a very good point my DB is 40k tracks and 6gb (it was 5gb in Rekordbox) luckily I chose exFAT and have had Zero issues.

The FAQ and manual should probably be updated to say that if you expect to have more than 20k tracks you should use exFAT

Or they could make the databases split at 4gb, im sure this could be done fairly easily with the database software they use.

Hi j141. Can I use exfat on the 8000 without this limit??

The manual for the 8000 says FAT16/FAT32 so possibly not, I would check with support, I use the SC5000’s

Aah damn, I got excited there for a minute.

im not familier with the 8000 is it possibe to have multiple usb devices or collections/crates to get arnoud the limits?

The 8000 can access 2 usb devices. Unfortunately I went & bought a 500gb ssd to store all my music on. So I only discovered this problem now when trying to run all my music through EP. Kind of a waste really :frowning:

the problem is solved informatically speaking, would have to request it as a feature of engine prime I guess, that this software creates a second database when it reaches the limit is possible, would also have to update the mcx8000 firmware to read it I guess, excuse me English

regards

[quote=“El_Corsario, post:8, topic:14147”] request it as a feature of engine prime I guess, that this software creates a second database when it reaches the limit is possible, would also have to update the mcx8000 firmware to read it I guess [/quote] I’m thinking the same thing.

Your english is fine. Thanks for the response.

1 Like

Actually you can have a database with 20k files on a FAT32 filesystem without exceeding the 4GB file size limitation, but it depends on the lenght of tracks and how much info has to be stored in the database. Ofcourse you can split your database on multiple usb storage devices or you can choose to format the drive in ex-FAT filesystem, but the main tasks here are knowing your equipment capabilities, patience, planning and preparation. It’s the concept I keep trying to promote with every opportunity.

In a previous post I opened a feature request for an up to date Engine user manual containing all fixes and features implemented since first release, complete with visual explanations of specific tasks and features that requires special attention. The goal is to obtain a well documented user experience both for the advanced and for the beginners. As well I opened a thread requesting the release of a service and repair manual for MCX8000 and for all Denon DJ equipment, preferably containing a spare parts catalogue specific for every device.

1 Like

@crazycraig When Jay_DenonDJ staff replied to you and bluntly suggested to format the drive to ex-FAT filesystem in order to avoid FAT32 filesystem file size limitations, do you think he made an assumption that you know how to use a computer and to work with various filesystems?:stuck_out_tongue_winking_eye:

He’s not making an assumption. He knows I can use a computer based on a number of things, like: I managed to install EP, transfer files & analyse them, hey, I’m actually responding on this forum using a computer!! (although I could be using my phone). It’s obvious that I’m “now” aware of this limit. Even Jay never thought about the p.db file being at it’s limit. What he did somehow manage to overlook is the fact that the mcx8000 cant use the exfat format. Perhaps you can enlighten me on why you think the 4gb limit should be common knowledge to anyone that’s used a computer? Or better yet, show me some documentation that tells guys with large libraries that a file called p.db might reach this limit, thus preventing further analysing & possibly cause EP to say ‘database corrupt’. Maybe I need some computer lessons, coz none of this was common knowledge to me.

Hey @crazycraig,

Thank you for you patience, I’ve escalated this to the tech team and should have some more info for you very soon.

Regards J

@crazycraig Regarding the FAT32 filesystem characteristics you can take a look at this table for comparison. http://www.ntfs.com/ntfs_vs_fat.htm

Regarding the p.db file size, it is not the file itself the problem, but the content stored in it (like track performance parameters, cues, waveforms) increasing its size every time you add a new track or perform edits on a track. The problem arrise when you want to work with FAT32 filesystem because the size of any file must not exceed 4GB, regardless of its extension. On FAT32 filesystem you have to be aware of file name and file path limitations also.

This info should defo be in the manual/faq and a warning should pop up in Prime if you are using FAT32 and you are approaching the 4gb limit

I actually do have more than 20K, it’s probably around 30K that the p.db file reaches it’s limit. Knowing your equipments capabilities is a really big deal, hence why I honestly believe that this is info that Denon should make their customers aware of. I specifically bought a new ssd for the sole purpose of having my entire library on one drive. I was having an issue where EP would no longer analyse my tracks & I couldn’t figure it out. I thought it was a software problem. I even asked here on the forum but no-one could understand why this was happening. I would actually still be scratching my head if it wasn’t for MixmasterG - The Denon Conversion Utility man, he actually pointed the problem out to me. So a big shout out to MixmasterG (DeCu) for helping out your fellow Denon brothers!!

Can’t you just untick the auto analyze option and then manually analyze selected crates/playlist?

That workaround was used by SC5000 owners when the linking option also struggled with larger databases.

Thanks Jay. I appreciate your help.

It is hard to believe that, after more than 2 years, Denon DJ software development team was not aware about FAT32 filesystem characteristics when they implemented only FAT16/FAT32 support for MCX8000 removable drives. They were well aware, but at the same time an assumption was made that MCX8000 will be used according to its intended design purpose - primary Serato controller and secondary standalone media player as backup so, as a result, it was not considered necessary to implement support for other filesystems, like ex-FAT.