Here’s the thing, though: I never saw Fleming’s Bond as a “brawler,” either. Based on Fleming’s description, he’s not that physically imposing, and there aren’t many passages in the books – if any – where he engages in an all-out, bone-shattering brawl. Certainly I can’t picture him picking up a sofa to clout a foe as he does in YOLT, or bursting through sheetrock as he does in CR.
Keep in mind that at the time of Dalton’s debut, we’d had three prior Bonds: Connery, who was physically larger than Fleming’s Bond and much more of a bruiser, Lazenby who was directed to mimic Connery but came and went fast enough that he didn’t make a huge impression on the general public and Moore, who was much less physical than either of them and more crafty than “scrappy.” So that’s 7 films with a “tougher” approach and 7 without. An even split. And the 7 “softer” ones had come immediately before Dalton’s. So he wouldn’t have been picked so much for his ability to beat people up, and it wouldn’t necessarily have been written into his scripts. It was only when we came full circle with Craig that the world seemed to decide that yes, James Bond is 50% defined by Fleming and 50% by Sean Connery, and the “best” Bonds are beefy hunks who throw down like soccer hooligans.
Ideally, an assassin shouldn’t have to engage in a lot of hand-to-hand combat, anyway. If he’s any good at his job, he should be able to kill quickly and efficiently from a safe distance. If it comes down to fists and knees at close quarters, I would think things have already gone pretty far sideways.