Conceptually the argument is exactly what Raymond wrote, but I want to cast it in the language of Peano arithmetic. Am I being as abstruse as possible simply to be difficult? Well, read the answers which relied on the English language, and you'll agree they did not help to make the conclusion any more sound. What I want to do is use as few concepts as possible, and no more than are needed to define '2+2=4', so each step of the proof is impossible to argue.
The cliffnotes version is that you define the natural numbers in the most bare-bones way possible; there exists an object called 'zero', and an ordered sequence of objects that come after it. More explicitly, 2 would be called S(S(0)), "the successor of the successor of zero" and 4 would be called "the successor of the successor of the successor of the successor of zero", or S(S(S(S(0)). Then you define the plus sign as a binary predicate which behaves the way we want it to. But this is where that formalism comes in handy; now we don't have to define x+y case-by-case for all possible arguments (in which case 2+2=4 would literally be by definition), but only by how + interacts with S( ) and 0.
If you're interested, you define a+0=0 and a+S(b)=S(a+b). Then, for instance, you can prove you get the successor of a number by adding one, which Raymond simply assumed:
a+1=a+S(0)=S(a+0)=S(a)
Notice that what I did had no bearing on the meaning of any of the symbols, but it simply followed from the rules I invented for substituting one string of symbols for another. Then the proof of 2+2=4 within this formal system just looks like a pencil-and-paper version of the Towers of Hanoi puzzle (with some parentheses missing)
SS0+SS0=
S(SS0+S0)=
SS(SS0+0)=
SS(SS0)=SSSS0