***If*** you are techy enough to run a VPS or a dedicated server with Linux....
...then for VPS I would go with Contabo. They are state-of-the art AND cheap and still their VPSs aren't over provisioned like so many other VPS providers. You also get "unlimited" traffic, which in their case means 32 TB/month which is FAR better than most others. They have hosting in the US, EU and Asia. Their support is great.
(as for AWS, it can become pretty nasty with what they charge if you need a lot of bandwidth)
...for a dedicated server and If you are happy with just having a server in the EU (Germany or Finland) I have used Hetzner for years and I have been very pleased with them. Their support is excellent. (check under their server auctions to find a cheap server).
Velia (which is owned by GoDaddy these days) have ok dedicated servers at ok prices. Nothing too fancy, their support is ok.
Server4you as a suppler of both VPS and dedicated servers, I would not even touch with a pliar. Their support sucks so much.
Right now my server (a VPS) is in the server & comm room in my house, but I am looking into potentially rent a dedicated server from OVH for North America. They have some super cheap servers in the Rise series (I am looking at a Rise-3 server), BUT if you do large scale email marketing then avoid them like the plague, you will be kicked out fast. I have never used them myself , so I am a bit hesitant due to what their rumor is. A backup space of 500+ GB comes with all their servers.
I am not into shared hosting at all, but I know how to handle Linux "good enough". If you have the skills then just let the rescue systems that are part of all the above hosting companies and install a LAMP server, then put ISPconfig3 (free web panel) on top of that, theer are scripts that handle all the installation. It works great!
For domain names, and if you want cheap domains that are a bit harder to handle in the Advanced DNS handling since they don't support templates, then NameCheap.
GoDaddy is better with their DNS handling, because there you can set up a template that will set much of what you need to do after renting a domain name, it's super easy to just adapt the template a bit. Their domain names are slightly more expensive.
Attention: "role based name" email prefixes such as support, webmaster, info, contact, forum, service, affiliate, admin, traffic, partners, marketing, etc., for our member accounts are not permitted. They are acceptable for a vendor listing, but not for the member email. This is due to bulk email processors scrubbing them for delivery due to extremely high bounce rates. Please apply for membership using a personalized email prefix. Something like email@example.com.
If we discover you signed up with a role based name we will give you only one message to change it. After 3 days your account will be removed if the email is not changed.