Below is a depiction of the expansion of the universe. It was created by the NASA/WMAP Science Team, with some modification by ‘Cherkash’, who then posted it to Wikipedia.

As we can see, this depiction of the universe shows its expansion over time. If seen as a cup, the ‘lip’ or opening at right is our present time, and the circle there is the width of the observable universe. Moving to the left, each circle represents a billion year shift into the past. Within a fraction of a second of the Big Bang at the far left, cosmic inflation expanded the universe to nearly its present size. The ‘Dark Ages’ was a time after the first light of the Big Bang became redshifted into the cosmic microwave background (CMB) - represented here as the blue-green ‘afterglow’ - and the newly formed hydrogen atoms emitted only radio waves; for at least 150 million years the universe was completely dark until gravity pulled the atoms into stars and nuclear fusion lit them. It still continued to expand, but the expansion began to accelerate about 4 or 5 billion years ago due to Dark Energy.

Where are we on this illustration? We are at the very center of the rightmost circle. A billion years ago ‘we’ were at the center of the next circle to the left. Our location relative to the observable edge of the universe would be a straight line down the center of the cup as we regress into the past, even though the solar system, the two suns that preceded our’s, and our galaxy would all disappear as we passed their times of origin.