When you asked for milk, Gojo went into his room, did something, and then handed you a glass. There's no way a refrigerator would be in Gojo's room.