You may want to try a different DAW to manage this project.
I don't know about other DAWs, but Reaper uses non-destructive editing for the audio files. This means that no matter how many changes you make to the audio, the original files never change. You will only need to save a single copy of the original audio files and version the project file (which is a simple text file).
When working with a large amount of tracks or effects you have the option to render portions of the audio to stem tracks to save processing power. This will create a new file, but it does not need to be put in version control since you can regenerate it at any time from the original files and the project file.
Edit:
To share a project all you would need to do is put all the files in Git. Any audio files will only go in once and will not change. The project files are very small text files so it doesn't matter much how often they change. This should allow you to share the files without using up huge amounts of bandwidth and storage capacity.
Edit:
From the Reaper User Guide:
All of the settings, preferences and custom files are text files.
Any of them can be opened and modified using a text editor such as Notepad. This even applies to REAPER’s
project files, with .RPP extension.
The project files are simply text files in an XML-like format. The project file contains the project properties and a list of tracks. Each track contains a list of items. Each item lists the original wav file, the start point and length of the clip, and other properties of the item. This type of file works really well in Git or any other source control repository. Since the large wav files associated with the project will only be added or removed and the small project files are the only files being changed it will minimize the amount of data that needs to be shared when a change is made.