How to git exclude source files on the webserver but keep it on all other repos updating all past commits

Question

I looked in many answers but I could make it work. I am working with Adobe Animate games, so the source files are big .FLA files, I noticed that if I could ignore these files (more than 600) on our server we could save almost half of the space there. So I put the *.fla and *.ai files inside the exclude on the server, but I believe it just working for new files, as this is a one year project all the files that already have referency will stay there. If I try to exclude them they will be excluded on Dev repos as well. Is there a way of remove the actual reference? Erase all the files from the server but keep them on Dev's repos? Thank you!

All the extensios are already inside .git/info/exclude on the server After that I exclude the files there, them the server ask me to commit, and all the files are erased, so I revert on my local, and when I pull into the server the files are back there.