Could be excessive wear in the steering column as the factory used ball bearings in the bottom but there is no lubrication since day one when it was assembled. They wear out and eventually fall out allowing a half inch or more of play. Rag joints may look solid but inside the rubber has eroded and is missing in action allowing a lot of play in the front end.
You said you replaced the tie rod ends and idler arm with a full front end rebuild. Those are the parts most frequently associated with a sloppy front end though worn ball joints can also cause issue (this is most evident in strange tire wear patterns).
You could have wear in your steering gear box, as I have worn out several in my Suburban(s) by driving them all over the country and back again (put 270,00 miles on one and well over 360,000 on it's new replacement; before buying a new Silverado in 2011 to replace it, as the steering wheel had so much slop in it that my wife said to park it, and go buy a new one.
It is possible to rebuild a steering gear box (requires a few specialized tools), but for the price you can not beat a remanufactured replacement.