Good questions:
- DVC: no new commands to learn (we extend Git) and you don't need S3.
- Git LFS: we inject useful views into your large files inside GitHub itself (in commits and PR's) unlike Git LFS (e.g. check this model diff: https://youtu.be/lAyymscJUvI?t=87), we scale to much larger sizes (100 terabytes), and we deduplicate better (Git LFS considers a 1 line change to a large CSV file a new entire file, our technique captures the differences)
In my experience, the lower you go....the model:
- hallucinates more (one time I asked Llama2 what made the sky blue and it freaked out and generated thousands of similar questions line by line)
- is more likely to give you an inaccurate response when it doesn't hallucinate
- is significantly more unreliable and non-deterministic (seriously, providing the same prompt can cause different answers!)
At the bottom of this post, I compare the 2-bit and 8-bit extreme ends of Code Llama Instruct model with the same prompt and you can see how it played out: https://about.xethub.com/blog/comparing-code-llama-models-locally-macbook