With regards to losing weight, gaining weight, or maintaining your weight, calories are the only thing that matters. But with regards to body composition and overall health, the “quality” of those calories definitely matters.
From a weight perspective the amount of energy you’re taking in matters foremost. Most people are going to weigh the same, assuming calorie output is equal, whether they eat 2500 calories of cookies or a balanced nutritious diet. There’s some tiny impacts from how much energy food takes to digest, but not large enough for most people to need to think about.
That said, you’ll sure feel a lot better, put less stress on your body, and find it easier to engage in healthy and/or enjoyable physical activities if you have a reasonably balanced nutritious diet than if you just eat an appropriate number of calories of junk food.