In short, the ladder describes the chain of thinking that leads us from observable facts (at the bottom) to making decisions and taking real-world actions (at the top).
Why it's useful is it provides a neat model of reference when two (or more) people find themselves in disagreement.
Say Luigi is keen to make more use of test-driven development (TDD), but Mario just wants to get on writing code. How can they reconcile their differences?
LUIGI: I read that TDD is The Magic Bullet and I'd like us to start using it! MARIO: No way! TDD is awkward and wastes time; your idea is bad.
Uh oh, looks like we have a conflict. Luckily Luigi's familiar with the Ladder of Inference, so he steps down a rung:
LUIGI: I think TDD results in better test coverage, and test coverage is important - do you agree? MARIO: Heck no; I'm a great programmer, and I can write tests for any edge cases afterwards - I'd rather concentrate on delivering solid code quickly.
Okay; it's progress but we've still not landed on common ground. Luigi and Mario have different ideas about the value of test coverage - but why?
LUIGI: I think test code coverage is a good way to maintain code quality, and I think quality is important to our customers - do you agree? MARIO: Of course I agree that code quality is important - our customers hate it when we introduce bugs. I just don't think that writing loads of useless tests is the way to deliver it!
Did you spot the consensus? Luigi and Mario both agree that quality is important. Of course they do; they're professionals. But importantly the conversation is now about quality and how it is delivered - this is a place where they can start to have a sensible discussion.
Unburdened by opposition they can talk about whether there really is a quality problem right now. If they don't know, they can talk about how they can find out. And if there is, they can talk about all the ways they might go about solving it, with TDD as one tool of many at their disposal.