[cvsnt] Re: CVSNT does not support *large* binary files (crashes onthem)

[Arthur Barrett]

Serge,

> Had tried the latest release (cvsnt-2.5.03.2260).
> 

The release I asked you to try is the latest testing release on cvsnt.
org which is close to being stable.  It's 2.5.03.2354.  But in this case
I do not think that there were any changes to this processing between
2260 and 2354.

> Had to reinstall back to cvsnt-2.5.02.2115 since changes in
> cvsnt-2.5.03.2260 has broke our CM scripts.

CVSNT is "free", not as in "free beer" but as in "freedom to
contribute", therefore when you come across a bug please create a new
thread in the newsgroup and report the bug and supply the necessary
information so that we can address it:
http://march-hare.com/cvspro/faq/faq2.asp#2z

> Please consider a situation of 1 byte binary file.
> Then consider 1 billion changes in this file.
> As a result file.bin,v will be *large* binary file.
> 
> The same logic is appliable to text files.

CM best practice informs us that if a file changes that many times then
it is likely to need refactoring. 

>> We recommend at least 2 x the size of the largest directory
>> (sum of all file sizes in that directory including dead files)
>> in available (real) memory if you are using -kkv and -kb,
>> significantly more if you are using -kB.
>
> Wow!
> Does it mean that CVSNT uses in-RAM processing?

Well you can rely on swap/page but the server will (probably) grind to a
halt.  Some OS's are better at this than others.

> Does it mean that if we are planning to grow to few tens
> of gigabytes in a single folder then 32-bit host OS is out 
> of question and we need 64-bit box with 2 x tens of gigabyte 
> of RAM?

A good reason to get support before designing your repository is so that
you can be informed that storing tens of gigabytes of files in a single
folder is poor CM and also technically unfeasible with many CM tools
including CVSNT.  Commercial support is best for commercial users and
gives you access to the engineers who write the software:
http://march-hare.com/cvspro/

We do support 64 bit HPUX (IA64 and PA RISC) and Solaris, I believe that
64 bit windows is planned for later this year - I do not know about 64
bit Linux.

The original CVS was certainly not designed to store large binary files,
however CVSNT probably does versioning of large binary files as good as
any other system.  

If you do not need multiple versions of these files then be sure to use
-k1 in cvswrappers and/or when you add the file to the repository
http://www.cvsnt.org/manual/html/Substitution-modes.html.

One possible enhancement would be for someone to add another mode where
multiple revisions are stored, but each revision is stored complete - no
deltas at all (-kb and -kB both use deltas, but different algorithms).
This is currently not a priority for March Hare Software, but could
change based on commercial customer feedback.

Regards,


Arthur Barrett