Use a CDN like Cloudflare and/or Amazon cloudfront for the media. Make sure your server (VPS) is properly sized with enough CPU and Ram.
Cloudflare’s DNS is my go-to which performs well. If using QUIC.cloud, their DNS seems to perform well. But if you registered your domain with someone like GoDaddy or NameCheap, you’re using their DNS by default which is usually slow.
https://www.webpagetest.org/ is a good resource to test your website with.
----
Asset Cleanup Pro plugin works great for improving wp site speed. When you edit a page it will show a list of all the CSS and JS that loads on that page and you can unload specific JS and CSS files on that individual page. OR you async or defer these files. You can also apply site wide rules to . (Example: unload wpforms js for all urls with /product/ )
Great documentation on it
An unload rule is the action that prevents a specific asset (CSS/JavaScript file) from loading on a certain page, a group of pages or site-wide (everywhere, on all pages of the website). These include: Unload on this page (applies only on the post/page that you’re editing) Unload everywhere...
assetcleanup.com
Also, in the settings >> Plugins area you can 'deactivate' plugins and all their JS CSS and HTML from specific pages or page types.
Be careful because you can unload or async a JS file that were important to the specific plugin you unloaded it from. But no worries just changing the setting to the original setting will fix that and nothing is broken anymore!
I don't know the differences in the caching plugins, but I use Litespeed cache with a Litespeed server one click WP install on Digital Ocean. The documentation is very clear and easy. I'm not very technical. You'd probably get a performance boost with a VPS if you're on shared hosting as well.