The cost of a line trace depends not only on “is it a line trace,” but also on how long the line is (10 units is much cheaper than 10000 units!) and how many objects “could” be hit (approximately, how many objects are in the axis-aligned bounding box of the line.)
So, long, diagonal, lines, across a battlefield: More expensive.
Short, straight down, lines, to find whether you have ground contact: Less expensive.
But, a long trace from the camera, to the sun, if the sun is above (and not on the horizon,) could be totally fine because there’s nothing to test against in the AABB.
In my head, I would estimate the cheapest line trace at 1 microsecond, and the most expensive reasonable line trace at 10 microsecond. If I want to spend 1 ms per frame on just line traces, then I can do 100 big traces, or 1000 small traces. I’m probably off by some factor, though, because I haven’t measured these things for many years, and, again, very long, very diagonal, traces through cluttered environments, are massively more expensive!