Clean Growlers immediately after use. This will prevent bacteria growth and the need for cleaning detergents that will leave residue that can harm the beer's flavor. Fill your growler with very hot water, shake and rinse. Repeat this process several times. Let dry upside down and store with cap off.
The fresher, the better. Think of your growler like a loaf of fresh bread. It will be ok several days later but will never be better than the first day. This has to do with the introduction of oxygen when filling. This is why you'll want to consume your growler immediately after opening as well. Oxygen will cause stale flavors. Loss of carbonation after opening will also dramatically affect taste. Damage from oxygen defeats the purpose of fresh draft beer.
Store filled growlers cold and keep away from light. Refrigeration will slow the negative effects of oxygen and darkness will prevent "skunking" caused by light damage