My understanding is that this is only kinda true in a roundabout way. I think besides the hidden age of decline, it's more to do with total skill points. The softcap for that is another hidden, but obviously faster learners tend to more likely be at their soft cap or close to it so faster learners tend to have more points before decline. Decline seems to be more severe for guys with a high total skillpoints.
So say you believe the decline is done by a percentage, like you lose a tiny % every day when in decline, but this is somewhat propped up by training and sparring so it's not as noticeable in some skills or at all early on. Someone with maximum skill points (say 2700, which is definitely on the higher end) would lose more points total in raw numbers each day (and it'd be shaved off everything) than someone with a much lower points total (like 2200).
Now I'm not saying it's definitely done by percentage or anything like that, but the proof is the relative longevity of specialist grapplers. My 43 year old grappler should definitely be in decline given his age, but he's dropped a total of 1 point (spread over all skills) in over 10 weeks. However, his total skill points are a relatively low amount, because he's got ONLY grappling and physical skills, and basically nothing at all in the others. Those skills though, are all 147+.... so basically 15++ including physicals. I have no idea how close this is to his actual overall softcap, but this trend of specialist grapplers (aka ones that don't also have a heap of points in striking stuff) being more immune to ageing seems to be fairly consistent and supports the idea that total skillpoints play a factor.
This is one of the few reasons why people caution against oversparring early and also especially building 'spreaders'. Couple of misplaced training sessions here and there, and they're all points that go towards your softcap even if you're realistically not actually making use of any of those. It's much easier to go from 1 to 10 accidentally in a skill you aren't focusing on than 141 to 150 in a skill you care about, but it counts the same towards the skillpoint softcap.
EDIT: But to address the original question, yes you have to train skills at least once every now and then (including physicals) otherwise it'll start to go down. Even if you don't see depops, it doesn't mean they're dropping due to lack of refreshment, it's just that he hasn't dropped below the threshold to depop. On creation with the max 110 points in a skill, you only JUST hit the wonderful mark, which is why even the slightest decline will show a depop