Search results are not showing up on external application

Issue Definition:

We are having issues with getting search results back from the Intranet zone of the Prod_alispasset web application which has a URL of https://alispasset.sharepoint.com and is using ADFS authentication. The results will display on the same web application when it is accessed using its default zone URL https://spalispasset.sp.com which uses NTLM authentication.

Symptoms:

1. SPSiteURL Managed Property always shows default zone url irrespective of the AAM Zone URL used for search.
2. Search results are working fine for existing files with Default zone URL with NTLM
3. Newly uploaded files are getting crawled where we see success crawls.
4. We could see the new results are coming with default search but not on the custom search.
5. We confirmed that we are able to get results using osssearchresults.aspx on each and every one of them
6. We noticed that the query sent while searching using osssearchresults.aspx and search.aspx are different. 
7. Once user is in session, bypassing Akamai, and f5 with a hosts file entry on the intranet zone does yield search results but URLs are from the default zone AAM

Root cause: 

In late 2017 Microsoft pushed a feature called AAM caching from SPO. The main benefit of AAM caching is increased query performance. Unfortunately, in Publishing\Consuming scenarios where AAM’s are not present. This causes the AAMs to be dropped.

Now, a few caveats (please pay attention):

• First, this technically isn’t a bug. It was an intentional design change to improve the performance of SharePoint. From the affected Customers point of view, the use of the word ‘bug’ may not matter, but sometimes it helps to know that MS did this intentionally.
• Microsoft Product Group is working on the coding to fix this whether this is something which they can be applied separately to publishing farms or on hotfix.
• We can use the PowerShell script to populate the URL Mapping Cache in the search farm. But we CANNOT prevent the cache from being cleared again. As per MS, the cache clears every 15 days or it may be sooner. We may see this problem again in 1 day or 15 days. 
• The fix will be the same…and you can script this to run nightly\weekly\etc if we’d rather not test how often the issue occurs

Following are the PowerShell commands we ran to populate the URL Mapping Cache in the search farm:

#add-pssnapin microsoft.sharepoint.powershell

#Add-type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.Office.Server.Search.dll"

#Get URL Mapping existing

$ssa = Get-SPEnterpriseSearchServiceApplication SP2013SSA

$tenantId = [Guid]::Empty

$zoneId = 1

$ssa.GetUrlMapping($tenantid,$zoneId).ForwardUrlMapping

$ssa.GetUrlMapping($tenantid,$zoneId).ReverseUrlMapping

#Set URL Mapping inlcuding existing. This command overwrites existing entries, so we need to include all, existing url mappings pulled from above commands. Below values are based on the today’s date. So, they may vary, and you need to update accordingly.

#Populate forward mappings variable

$ssa = Get-SPEnterpriseSearchServiceApplication

$tenantId = [Guid]::Empty

$zoneId = 1 # Intranet zone

$forwardMappings = New-Object 'system.collections.generic.dictionary[string,string]'

$forwardMappings.Add("https://spalispasset.sp.com","https://alispasset.sharepoint.com")

$forwardMappings.Add("https://spcontoso.sp.com","https://contoso.sharepoint.com")

$forwardMappings.Add("https://spwww.sp-america.sp.com","https://www.sp-america.com")

$forwardMappings.Add("https://spwww.sp-london.sp.com","https://www.sp-london.com")

#Populate reverse mappings variable

$reverseMappings = New-Object 'system.collections.generic.dictionary[string,string]'

$reverseMappings.Add("https://alispasset.sharepoint.com","https://spalispasset.sp.com")

$reverseMappings.Add("https://contoso.sharepoint.com","https://spcontoso.sp.com")

$reverseMappings.Add("https://www.sp-america.com","https://spwww.sp-america.sp.com")

$reverseMappings.Add("https://www.sp-london.com","https://spwww.sp-london.sp.com")

#Update URL mappings

$urlMapping = New-Object 'Microsoft.Office.Server.Search.Query.UrlMapping.UrlMapping'

$urlMapping.ForwardUrlMapping=$forwardMappings

$urlMapping.ReverseUrlMapping=$reverseMappings

$ssa.UpdateUrlMapping($tenantId,$zoneId,$urlMapping)

#To Update 2nd zone:

$ssa = Get-SPEnterpriseSearchServiceApplication

$tenantId = [Guid]::Empty

$zoneId = 2

$forwardMappings = New-Object 'system.collections.generic.dictionary[string,string]'

$forwardMappings.Add("https://spalispasset.sp.com","https://public.alispasset.sharepoint.com")

$forwardMappings.Add("https://spwww.sp-canada.sp.com”, “https://www.canada.com”)

$urlMapping = New-Object 'Microsoft.Office.Server.Search.Query.UrlMapping.UrlMapping'

$urlMapping.ForwardUrlMapping=$forwardMappings

$ssa.UpdateUrlMapping($tenantId,$zoneId,$urlMapping)