What is the universe expanding into?

Hubble first discovered that universe is not static and is expanding. Ever since I knew this, a question bugged me - What is the universe expanding into? This is a natural question following the fact that the universe is expanding because in daily life we see that as objects expand, they take up more space. So, what is this additional space that the universe is expanding into?

I came across several versions of answers to this question but none to my satisfaction. I finally found the one that satisfied me. Before coming to the answer that I found satisfactory, let us summarize the variety of answers to this question.

1. The universe is defined this way: We define the universe as the one occupying our entire space. So, by definition, the expanded part becomes part of our universe. Hence, there is no meaning to the question, what it is expanding into! 

You can clearly see why this is unsatisfactory. This is just playing with the semantics.

2. Similar to addition to infinity: The infinity explanation says that expansion of the universe (adding something to the universe) is similar to adding something to infinity that again gives infinity. So, you get the "universe" after expansion, which is practically the same. Hence, the question of expansion is mathematically well explained.

This again, like the definitional approach is unsatisfactory!

3. Expanding balloon: This is the most popular explanation. It says that expanding universe can be thought of increasing distance between two points on an expanding balloon.

While this gives us some idea, it still doesn't address our core question - balloon, in this case, is still expanding into something, which is increasing the distance between the points on the balloon. What is it that the universe is expanding into, that is increasing the distance between the points in the universe?

4. Space is created as it expands: This explanation says that there is a difference between the "expansion" that we usually see in the daily world and the "expansion of the universe". In our daily life, the object that is expanding is embedded in space, and it occupies more space as it expands.

However, in the case of the universe, it is not embedded in a "space" that it can expand into. The universe itself is "space". The "space" itself is expanding. It creates more space as it expands. The newly created space then becomes part of our universe, as per our definition.

This explanation brings out the important distinction between expanding objects in space and expansion of the universe. Though it is satisfactory to a large extent, the expansion phenomenon still seems mysterious.

5. Geometrical explanation: I found the following answer of Adam Solomon satisfying

"It's not expanding "into" anything. Like all of the curved spacetimes we talk about in general relativity, the spacetime describing an expanding universe isn't embedded in some higher-dimensional space. Its curvature is an intrinsic property. 
To be specific, it's the property describing how we measure distances in spacetime. Think about the simplest example of a curved space: the surface of a sphere. If I give you the longitudes of two points and tell you they're at the same latitude (same distance from the equator) and I ask you to tell me how far apart they are, can you do it? Not without more information: those two points will be much further separated if they're near the equator than if they're near the North or South Pole. The curvature of this space means that distances are measured differently at different points in space, particularly, at different latitudes. 
An expanding universe is also a curved space(time), but in this case the curvature doesn't mean that distances are measured differently at different points in space, but at different points in time. The expansion of the Universe means quite simply that the distances we measure between two points which are otherwise stationary grows over time. In effect, the statement that "space" is expanding is really a statement that our cosmic rulers are growing."
Let us understand this. 

First, let us consider a grid of parallel lines in 2-D space. They look as below.

We note that the distance between any two points "A-B", "C-D", "X-Y" is the same. 

Now, let us "bend" the 2-D grid into a 3-D sphere. The vertical lines become longitudes and horizontal lines become latitudes. Notice the change in an important property of the distance between the points. The distance between "A-B" is not the same as the distance between "C-D", which is not the same as the distance between "W-X". If "AB" is the equator, the distance between the points on the same longitudes decreases as we move further away from the equator. EF < CD < AB.

In other words, the curvature of space along the 3rd dimension (creating a sphere) changed the distance between the points as we move along the curvature.

Expansion of the universe is similar to this. The universe is not in a 3-D space as a sphere. It is a 4-D space-time, with time being the extra dimension. What we call as the expansion of the universe is the change in "distance between points" with "time" in the 4-D space-time. 

Just like the curvature of space in 3rd dimension in the case of a sphere causes the change in distance between points as we move along the curvature, the curvature of the 4th dimension, the time, changes the distance between points as we measure along the time (curvature of time). i.e., the distance between points in 3D space increase with time.

As we notice, it is a geometrical property of 4D space-time. It doesn't need anything for the space to expand into.