Recently I had to remove a submodule from our project which was achieved in easy way:

# Remove the submodule entry from .git/config
git submodule deinit -f SUBMODULE_DIRECTORY

# Remove the submodule directory from the superproject's .git/modules directory
rm -rf .git/modules/SUBMODULE_DIRECTORY

# Remove the entry in .gitmodules and remove the submodule directory located at SUBMODULE_DIRECTORY
git rm -f SUBMODULE_DIRECTORY