If you ever manually remove files from a git repository you have to tell git they have been removed and add the files to the index to commit the changes afterwards:

for FILE in $(git ls-files --deleted);do git rm $FILE;done