Tag Archive for: Semrush site audit tips

How to fix: Robots.txt not found

/by

Issue: Without a robots.txt file, search engines may crawl unnecessary or sensitive content, wasting crawl budget and potentially exposing private information.

Fix: Create a robots.txt file to guide search engines on what to crawl and what to ignore.

How to Fix for Beginners

  1. Create a robots.txt File: Add a plain text file named robots.txt to your site’s root directory (e.g., https://example.com/robots.txt).
      • Example: 
        User-agent: *
Disallow: /private/
Allow: /
  2. Specify Directives: Use Disallow to block specific files or directories and Allow to specify what can be crawled.
    • Example: Block search engines from crawling your admin area with:Disallow: /admin/
  3. Test the File: Use Google’s robots.txt Tester in Google Search Console to ensure your file works correctly.
  4. Upload to Root Directory: Place the robots.txt file in the main directory of your website so search engines can find it.
    • Example: Upload the file to https://example.com/robots.txt.
  5. Monitor for Issues: Regularly review and update the file as your site structure changes.

Tip: A well-configured robots.txt file optimizes crawl efficiency and protects sensitive content.

 

More articles relating to Robots.txt file:

How to fix: Pages with more than one H1 tag

/by

Issue: Having multiple <h1> tags on a page might confuse users and search engines about the page’s main topic, even though it’s technically allowed in HTML5.

Fix: Use a single <h1> tag per page for the main heading and use <h2> to <h6> tags for subheadings.

How to Fix for Beginners

  1. Identify Extra <h1> Tags: Check your page’s HTML for multiple <h1> tags.
    • Example: <h1>Main Heading</h1> and <h1>Another Heading</h1> are redundant.
  2. Keep Only One <h1>: Select the most important heading to remain as the <h1> tag.
    • Example: <h1>Main Heading</h1> stays, while others change.
  3. Use Subheadings for Structure: Replace additional <h1> tags with <h2> to <h6> tags based on their importance.
    • Example: <h2>Subheading 1</h2> or <h3>Subheading 2</h3>.
  4. Maintain a Logical Hierarchy: Ensure headings flow logically (e.g., <h1> → <h2> → <h3>) to make your content easier to read.

Tip: A single <h1> improves clarity and SEO while keeping your page organized for users and search engines.

How to fix: URLs longer than 200 characters

/by

Issue: URLs longer than 200 characters are not SEO-friendly. They look intimidating to users, reducing clicks, shares, and overall usability, which hurts your SEO performance.

Fix: Shorten your URLs to a reasonable length while keeping them clear and descriptive.

How to Fix for Beginners

  1. Identify Long URLs: Use an SEO tool to find URLs over 200 characters.
    • Example: https://example.com/category/subcategory/specific-page-title-with-too-many-words-and-details
  2. Simplify the Structure: Remove unnecessary words, parameters, or folder levels.
    • Example: Change https://example.com/category/subcategory/this-is-a-long-title-that-needs-shortening to https://example.com/category/short-title.
  3. Use Descriptive Keywords: Keep the URL meaningful and focused on the page’s topic.
    • Example: Replace https://example.com/p=123456789 with https://example.com/product-name.
  4. Set Up Redirects: If you shorten an existing URL, implement a 301 redirect from the old URL to the new one.
    • Example: Redirect https://example.com/very-long-url to https://example.com/short-url.
  5. Avoid Dynamic Parameters (If Possible): Replace unnecessary URL parameters with static, readable keywords.
    • Example: Change https://example.com/page?id=123&ref=abc to https://example.com/page-name.

Tip: Short, clean URLs are easier for users to remember and improve your click-through rate and SEO.

How to fix: Subdomains don’t support HSTS

/by

Issue: Subdomains without HTTP Strict Transport Security (HSTS) may serve unsecured content, potentially exposing your users to risks and harming trust and security.

Fix: Configure your server to support HSTS for your subdomains to ensure all content is served securely over HTTPS.

How to Fix for Beginners

  1. Check Server Support: Confirm that your hosting provider or server configuration supports HSTS.
  2. Enable HSTS for Subdomains: Add the HSTS header to your server configuration and include the includeSubDomains directive.
    • Example: Strict-Transport-Security: max-age=31536000; includeSubDomains
  3. Test Implementation: Use tools like SSL Labs to verify that HSTS is properly enabled across your subdomains.
  4. Update DNS Records (Optional): If your domain supports HSTS preload, ensure it is added to the preload list for maximum protection.
  5. Monitor for Issues: Check regularly to ensure all subdomains are being served securely with HSTS enabled.

Tip: Implementing HSTS improves security by ensuring all content is served over HTTPS, building user trust and protecting sensitive data.

How to fix: Outgoing external links containing no follow attributes

/by

Issue: The nofollow attribute on outgoing external links tells search engines not to follow the link, which means no SEO value (link juice) is passed to the linked page. Unintentionally using nofollow may affect your SEO efforts and the crawling process.

Fix: Check your external links and remove nofollow attributes where they’re unnecessary or used by mistake.

How to Fix for Beginners

  1. Identify Nofollow Links: Search for <a> tags in your code that contain rel="nofollow".
    • Example: <a href="https://example.com" rel="nofollow">Visit Example</a>
  2. Decide If Nofollow Is Needed:
    • Keep nofollow for links to untrusted sources, paid content, or affiliate links.
    • Remove nofollow for credible external links that enhance user experience and add value.
  3. Edit the Link Tags: Remove the rel="nofollow" attribute for links you want to pass link juice.
    • Example: Change <a href="https://trustedsite.com" rel="nofollow">Trusted Site</a> to <a href="https://trustedsite.com">Trusted Site</a>.
  4. Audit Regularly: Use SEO tools to scan for nofollow attributes in outgoing links and review them periodically.

Tip: Use nofollow strategically to balance SEO benefits and compliance with linking policies.

How to fix: Pages that need more than 3 clicks to be reached

/by

Issue: Pages that require more than 3 clicks to reach from the homepage are less likely to be visited by users or crawled by search engines, reducing their visibility and impact on SEO.

Fix: Adjust your internal linking structure so important pages are no more than 3 clicks away from the homepage.

How to Fix for Beginners

  1. Identify Deep Pages: Use an SEO tool or sitemap to find pages that require more than 3 clicks to access.
    • Example: https://example.com/category/subcategory/page might be too deep.
  2. Add Internal Links: Link these deep pages from higher-level pages closer to the homepage.
    • Example: Add a link to the page in a category menu or a popular articles section.
  3. Use Navigation Menus: Include important pages in dropdown menus, sidebars, or footer links for easier access.
    • Example: Place links to “Services” or “Products” directly in the main navigation bar.
  4. Review Site Structure: Simplify your site’s hierarchy to make critical pages easier to reach without creating too many levels.
  5. Test Click Depth: After updates, navigate your site like a user to ensure essential pages are reachable within 3 clicks.

Tip: Shallow site structures improve SEO and make it easier for users and search engines to find your content.

How to fix: Links with non-descriptive anchor text

/by

Issue: Non-descriptive anchor text like “click here” or “right here” doesn’t tell users or search engines what the linked page is about, reducing its SEO value and ranking potential.

Fix: Replace vague anchor text with clear, descriptive text that reflects the content of the linked page.

How to Fix for Beginners

  1. Find Non-Descriptive Anchors: Look for links with generic anchor text like “click here” or “read more.”
    • Example: <a href="https://example.com">Click here</a>
  2. Write Meaningful Anchor Text: Update the anchor text to describe the linked page’s content.
    • Example: Change “Click here” to <a href="https://example.com">Learn About SEO Basics</a>.
  3. Use Keywords Wisely: Include relevant keywords in the anchor text, but ensure it flows naturally for readers.
    • Example: Instead of “Check this out,” use “Explore Our SEO Starter Guide.”
  4. Keep It Concise: Aim for short, meaningful text that is easy to read and informative.
    • Example: “SEO Best Practices” instead of “Click here to read our SEO guide.”
  5. Review Internal and External Links: Ensure all anchor texts, both internal and external, are descriptive and relevant to their destinations.

Tip: Clear anchor text improves usability, enhances SEO, and boosts click-through rates by providing context about the linked content.

How to fix: Links to external pages or resources returned a 403 HTTP status code

/by

Issue: External links on your site return a 403 HTTP status code, meaning access to the linked page or resource is forbidden. This often happens when a server blocks crawlers or restricts access.

Fix: Check the availability of the linked page or resource and take appropriate action.

How to Fix for Beginners

  1. Test the Link: Click the link in your browser to see if it works for users.
    • Example: If https://example.com/resource shows a “403 Forbidden” error, it’s not accessible.
  2. Verify in Google Search Console: Check if search engines encounter the same 403 error for the link.
    • Example: Use the URL Inspection tool in Search Console to test the link.
  3. Resolve with the Website Owner:
    • If the content was removed, contact the external site owner to confirm if it can be restored.
    • If crawlers are blocked, ask the owner to adjust their server settings to allow bot access.
  4. Update or Remove Links: If the issue persists:
    • Replace the link with a working alternative if the content is available elsewhere.
    • Remove the link if it’s no longer useful or relevant.
  5. Monitor for Recurrences: Periodically audit external links on your site to catch and fix similar issues.

Tip: Regularly checking and maintaining external links ensures a better user experience and keeps your site SEO-friendly.

How to fix: Pages with only one incoming internal linksource formatted as page link

/by

Issue: Pages with only one incoming internal link are hard for users and search engines to find. This reduces their visibility and lowers their chances of ranking in search results.

Fix: Add more internal links pointing to these pages from other relevant pages on your site.

How to Fix for Beginners

  1. Identify Low-Linked Pages: Use an SEO tool to find pages with only one internal link pointing to them.
    • Example: Your “FAQ” page has only one link from your “Contact” page.
  2. Add Links from Related Content: Link to these pages from other related articles or sections of your website.
    • Example: Add a link to the “FAQ” page from a product details page or a help article.
  3. Include Links in Menus or Navigation: Add links to important pages in your navigation menus, footers, or sidebars for better visibility.
    • Example: Include “FAQ” in your main navigation bar.
  4. Create Contextual Links: Use meaningful anchor text when linking to the page, describing its content.
    • Example: Instead of “Click here,” use “Visit our FAQ page for answers.”
  5. Test for Usability: Ensure the added links improve user experience without cluttering your content.

Tip: More internal links to key pages increase their visibility, SEO ranking potential, and accessibility for users.

How to fix: Too long link URLs

/by

Issue: URLs longer than 2,000 characters may not be supported by some browsers and can cause issues with crawling, potentially being ignored by search engines.

Fix: Shorten URLs to below 2,000 characters to ensure compatibility and improve crawlability.

How to Fix for Beginners

  1. Identify Long URLs: Use SEO tools to find links longer than 2,000 characters on your site.
    • Example: https://example.com/category/page?param1=value1&param2=value2... (too long).
  2. Simplify the URL Structure:
    • Remove unnecessary query parameters and session IDs.
    • Replace complex dynamic URLs with clean, static ones.
    • Example: Change https://example.com/page?ref=abc123&session=456 to https://example.com/page.
  3. Use URL Shortening for External Links: For extremely long external links, use a URL shortener (e.g., Bitly or TinyURL) while retaining descriptive anchor text.
    • Example: Shorten https://longexample.com/very/long/link to https://bit.ly/short-link.
  4. Check Internal Links: Review your site’s navigation and link architecture to avoid linking to overly long URLs internally.
  5. Test Shortened Links: After updating, ensure the new URLs are functional and still lead to the correct destinations.

Tip: Clean, short URLs are more user-friendly, SEO-friendly, and easier for search engines to process.