The name of the color khaki coined in British India comes from the Hindustani language (itself a borrowed form of the Persian word khak meaning dust), meaning "dusty, dust covered or earth colored." It has been used by many armies around the world for uniforms.
In Western fashion, it is a standard color for smart casual dress pants (trousers) for civilians.
Anyway, one does not necessarily need to be a civil to wear khaki, right?
Go neutral, go natural, add some military flavour to your wardrobe :)