There is tons for PVE. Like so many dungeons both public and private. Public dungeons get harder and harder the more people in them. But most you can solo and have fun. There are a few epic public dungeons that are hard enough you need to group up with the rando's you find to get through them. (Thanks to the way the mega server works, you'll always find a few people to hang out with). Plus you get some cool achievements for clearing these.
For the main dungeons you can use dungeon finder to find a group. Once you clear the dungeon for a certain level bracket within your own alliance, you unlock the other two faction's alliances and can then instantly teleport to these new dungeons.
Then there are some amazing guild quest lines, tons of crafting (everyone can craft everything, but its' still balanced.), soul shards to find, lore books to find, and some other crazy things.
I haven't reached veteran content yet, but there is so much to do PVE wise that I haven't had much time to PvP yet.