Harry and David climbed into the back. Harry pulled off his mask while David shoved the cash under the passenger's seat. "What took you so long, Chuck?"

Either or both of them could have asked Chuck that question. You could create a new paragraph for the speech, separate the characters actions into two paragraphs, refer to only one by name so that the named one becomes the focal point/subject, or leave it intact, etc... but no matter what you'll need to clarify. It could be as simple as he said/the taller brother asked/etc or you could shuffle things around with a re-write like this:

Harry and David climbed into the back. Harry pulled off his mask while David shoved the cash under the passenger's seat and leaned forward against the driver's shoulder. "What took you so long, Chuck?"

Now, it's not 100% crystal clear, but we have David taking an action on the driver, which cues the reader in that he's likely the one addressing Chuck. There are multiple other ways to write this that are even more specific, and that's the beauty of it!

What matters here is that you clarify.

The last thing you want is for your story to look like it's just a bunch of talking heads.

Dialogue tags give you a pause in speech without having to say a pause occurred. They can also help make your dialogue sound natural, provide the chance to insert a bit of emotional impact, or tension.

"Thank you for the rose, Jimmy," Clara said, "but I already have a prom date."

"Well," Noah mumbled as he fought every instinct to stare, "I've never had a female doctor before."

"Oh, Tom," she said, her fingernails tiptoeing along his spine. "If that was dinner, I can't wait for dessert!"