Yeah, they spend an entire night huddled up in a room on the planet, apparently without once calling the Enterprise to check what's going on or to report in.

Figuring out that the society is being run by a computer, with essentially no clues whatsoever beyond the behaviour of the Lawgivers, is a brilliant piece of Bat deduction.

And this is the first time that Starfleet's Prime Directive is mentioned in the history of Star Trek, and Kirk promptly ignores it approximately 12 seconds later.

Transcript

1
Captain's log, Stardate 3156.2. Lieutenants Sulu and O'Neill returning from Beta III, where they went with no security team to investigate the disappearance of starship Archon 100 years ago.
Kirk: Sulu, where's O'Neill?
Sulu: You. You're not of the Body.
2
Kirk: Get Dr McCoy!
Sulu: Landru... Landru... It's wonderful.
Kirk: Sulu! Where's O'Neill?
Sulu: Paradise. Paradise.
Kirk: See, this is why the captain needs to beam down into dangerous situations.
3
Caption: A larger team beams down:
Spock: They all look blank and contented.
Bilar: Visitors for Festival? It starts soon.
Kirk: Like Oktoberfest? Count me in!
4
Caption: The clock strikes 6; rioting and licentious behaviour breaks out:
Spock: Captain, do you need help fending off that woman kissing you?
Kirk: No, no... I can handle this...
5
Kirk: We're visitors, we need a room.
Hacom: You should be at Festival! Are you of the Body?
Reger: Hush, Hacom, they need shelter.
Kirk: Yes, to sit around uncomfortably and not contact our ship for the next 12 hours.
6
Caption: Next morning:
Reger: You did not attend Festival? Are you Archons?
Kirk: Maybe... did you owe them money?
7
Lawgiver: Come with us. You will be absorbed into the Body. It is the word of Landru.
Kirk: No.
Lawgiver: Uh... okay. You may run away while we decide what to do next.
8
Kirk: Scotty, what's going on?
Scotty: {over communicator} We're under attack! It started hours ago!
Kirk: Why didn't you call me?
Scotty: {over communicator} You didn't call us. We though you were upset with us...
9
Caption: They are captured and thrown in a dungeon:
Kirk: Time for some deduction.
Spock: The culture is stagnant because it is run by an all-powerful computer...
Kirk: ... which enforces contentment, but without any creativity!
Spock: Holy Landru, Captain.
Kirk: Exactly, Robin! ... I mean, Spock.
10
Marplon: I am to absorb you into the Body, but I'm part of a secret underground resisting Landru.
Kirk: Cool. We'll destroy Landru.
Spock: Destroying the computer will violate Starfleet's Prime Directive of non-interference.
Kirk: That only applies to people who aren't me.
11
Landru: {a computer} I am Landru. I maintain the good of the Body.
Kirk: But this culture sucks! Therefore you are evil. Therefore you must destroy yourself!
Landru: Oh. I hadn't considered it that way.
SFX: Bang!
12
Kirk: Well, we've destroyed the thing keeping your society from self-destructing. Good luck, see you round!
Marplon: Buh...