SharePoint 2013 Search - Remove an index component

Make sure that the current active topology is healthy and that the search component that you are about to remove is Active.

Note: If you have more than one active index replica for an index partition, you can remove an index replica by performing the procedure Remove a search component in the article Manage search components in SharePoint Server 2013.

You cannot remove the last index replica of an index partition using this procedure. If you have to remove all index replicas from the search topology, you must remove and re-create the Search service application and create a completely new search topology that has the reduced number of index partitions.

Step #1: Get the Search Service Instance and Start
$ssi4 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP04"

Step #2: Wait for the Search Service to Come Online
Start-SPEnterpriseSearchServiceInstance -Identity $ssi4

Step #3: Clone the Active Search Topology
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active

#To see which components you have, use the following command.
Get-SPEnterpriseSearchStatus -SearchApplication $ssa

Step #4: Get the ID of the Index Component to Remove
$indexComponentID = (Get-SPEnterpriseSearchComponent -SearchTopology $clone -Identity IndexComponent3).componentID

Step #5: Remove the original Index Component
Remove-SPEnterpriseSearchComponent -Identity $indexComponentID.GUID -SearchTopology $clone -confirm:$false

#Repeat step #4 and #5 for any additional servers you want to remove the index or any other component(like Admin,Query,Contentprocessing..etc) for.

Step #6: Activate Search Topology Again
Set-SPEnterpriseSearchTopology -Identity $clone

Step #7: Monitor the Distribution of the Index 
Get-SPEnterpriseSearchStatus -SearchApplication $ssa

SharePoint 2013 Search - Add Index partition and Index replica

For each 10 million items in the search index, you have to add a new index partition. When you add a new index partition, the search index has to be re-partitioned. Depending on the size of the search index, this re-partitioning can take several hours to complete.

Ref.. https://technet.microsoft.com/en-us/library/jj862355.aspx#Add a new index partition 

You add an index replica to the search topology to achieve fault tolerance for an existing index partition. You place the index replicas on separate failure domains on separate servers. When you add an index replica, you add a new index component to the search topology and associate it with the index partition that you want to make a replica of.

Step #1:Get Search Service Instance and start
$ssi3 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP03"
$ssi4 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP04"
Start-SPEnterpriseSearchServiceInstance -Identity $ssi3
Start-SPEnterpriseSearchServiceInstance -Identity $ssi4

Step #2:Wait for Search Service Instance to come online
Get-SPEnterpriseSearchServiceInstance -Identity $ssi

Step #3:clone the Active search topology
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active

Step #4: Create new index component
For example: If you have an existing index partition 0 with index replicas on App01 and App02 B, and you want to add a new index partition with index replicas on App03 and App04:

New-SPEnterpriseSearchIndexComponent –SearchTopology $clone -SearchServiceInstance $ssi3 -IndexPartition 1 -RootDirectory "D:\SPIndex"
New-SPEnterpriseSearchIndexComponent –SearchTopology $clone -SearchServiceInstance $ssi4 -IndexPartition 1 -RootDirectory "D:\SPIndex"

Step #5:Activate the cloned search topology
Set-SPEnterpriseSearchTopology -Identity $clone

Step #6:Verify that your new topology is active and that the index component representing the new index replica is added.
Get-SPEnterpriseSearchTopology -Active -SearchApplication $ssa

Step #7:Monitor the Distribution of the Index
Get-SPEnterpriseSearchStatus -SearchApplication $ssa -Text

Repeat this command until all search components, including the new index component, output the state Active. For a large search index, this could take several hours.

Create and Configure SharePoint 2013 Search service application

When you first create the Search Service Application, the search topology by default has all search components assigned to the server which is running the Central Administration. In multi servers farm scenario you need to change this topology using PowerShell.

Search architecture that you choose depends on how much content has to be searchable:

You can refer the below diagram to have better understanding for the topology that we are about to configure for 0-20 million items:

In this environment, we have five servers: two Front-Ends, two Application servers and one Index server.

Server Names:

• CA1VMSPUATAPP01
• CA1VMSPUATAPP02
• CA1VMSPUATAPP03
• CA1VMSPUATAPP04

Below are the steps that we are going to perform:

• Create Search Service Application and Proxy
• Add a New Index Component
• Add a New Crawl Component
• Add a New Content Processing Component
• Add a New Analytics Component
• Add a New Admin Component

We have assigned the following rules to each:

• App01, App02 server are responsible for Query Processing and Index partitions.
• App03, App04 has the Admin, Crawler, Content Processing and Analytics Processing roles.

#Load the SharePoint PowerShell snap-in or run these commands at the SharePoint Management Shell
Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

#How to create a search service application.
$ssiAPP01 = "CA1VMSPUATAPP01"
$ssiAPP02 = "CA1VMSPUATAPP02"
$ssiAPP03 = "CA1VMSPUATAPP03"
$ssiAPP04 = "CA1VMSPUATAPP04"

$SearchAppPoolName = "Search_Service_AppPool"
#You’ll need to change the default content crawl account. By default Search uses the Farm account, which is a really bad idea. 
$SearchAppPoolAccountName = "corp\ca1_uat_searchpool"
$SearchServiceName = "Search Service Application"
$SearchServiceProxyName = "Search Service application proxy"
$DatabaseName = "SearchService_DB"

#Create a Search Service Application Pool
$spAppPool = New-SPServiceApplicationPool -Name $SearchAppPoolName -Account $SearchAppPoolAccountName -Verbose

#Start Search Service Instance on all Servers
Start-SPEnterpriseSearchServiceInstance $ssiAPP01 -ErrorAction SilentlyContinue
Start-SPEnterpriseSearchServiceInstance $ssiAPP02 -ErrorAction SilentlyContinue
Start-SPEnterpriseSearchServiceInstance $ssiAPP03 -ErrorAction SilentlyContinue
Start-SPEnterpriseSearchServiceInstance $ssiAPP04 -ErrorAction SilentlyContinue

#Create Search Service Application
$ServiceApplication = New-SPEnterpriseSearchServiceApplication -Partitioned -Name $SearchServiceName -ApplicationPool $spAppPool.Name -DatabaseName $DatabaseName 

#Create Search Service Proxy
New-SPEnterpriseSearchServiceApplicationProxy -Partitioned -Name $SearchServiceProxyName -SearchApplication $ServiceApplication

Search service application has been created. Now, we need to configure different search components as described above and then finalize the search topology.

#Clone the active search topology
The command creates a clone search topology that can be referenced with $clone if you continue to use the same SharePoint 2013 Management Shell to add or remove search components and to activate the search topology.

$clone = $ServiceApplication.ActiveTopology.Clone()

#Get Search Service Instance
$ssiapp01 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP01"
$ssiapp02 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP02"
$ssiapp03 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP03"
$ssiapp04 = Get-SPEnterpriseSearchServiceInstance -Identity "CA1VMSPUATAPP04"

#We need two admin components
New-SPEnterpriseSearchAdminComponent –SearchTopology $clone -SearchServiceInstance $ssiapp03
New-SPEnterpriseSearchAdminComponent –SearchTopology $clone -SearchServiceInstance $ssiapp04
 
#We need two content processing components for High Availability
New-SPEnterpriseSearchContentProcessingComponent –SearchTopology $clone -SearchServiceInstance $ssiApp03
New-SPEnterpriseSearchContentProcessingComponent –SearchTopology $clone -SearchServiceInstance $ssiApp04
 
#We need two analytics processing components for High Availability
New-SPEnterpriseSearchAnalyticsProcessingComponent –SearchTopology $clone -SearchServiceInstance $ssiApp03
New-SPEnterpriseSearchAnalyticsProcessingComponent –SearchTopology $clone -SearchServiceInstance $ssiApp04
 
#We need two crawl components for High Availability
New-SPEnterpriseSearchCrawlComponent –SearchTopology $clone -SearchServiceInstance $ssiApp03
New-SPEnterpriseSearchCrawlComponent –SearchTopology $clone -SearchServiceInstance $ssiApp04
 
#We need two query processing components for High Availability
New-SPEnterpriseSearchQueryProcessingComponent –SearchTopology $clone -SearchServiceInstance $ssiApp01
New-SPEnterpriseSearchQueryProcessingComponent –SearchTopology $clone -SearchServiceInstance $ssiApp02

As per the plan we have created all the search components.

Assuming that you do not have have index partition, and that partition has no replicas.  So let’s look first at adding a new search partition.

Note: To scale out the search index, you add a new index partition for each 10 million items.

To achieve fault tolerance for the search index, you add an index replica of an existing index partition to the search topology. Each index replica contains the same information.

#We need two index partitions 0,1 (0-20 million items)
#If you have 0-10 million items then 
New-SPEnterpriseSearchIndexComponent –SearchTopology $clone -SearchServiceInstance $ssiApp01 -IndexPartition 0 -RootDirectory "E:\SPIndex"
New-SPEnterpriseSearchIndexComponent –SearchTopology $clone -SearchServiceInstance $ssiApp02 -IndexPartition 1 -RootDirectory "E:\SPIndex"

#Activate the Topology
$clone.Activate()

# Verify the Topology 
$ssa = Get-SPEnterpriseSearchServiceApplication
Get-SPEnterpriseSearchTopology -Active -SearchApplication $ssa

# Verify the diagnostic information for all search components in the active topology of a Search Service Application.
Get-SPEnterpriseSearchStatus -Text -SearchApplication $ssa

That's it. Our search service application configuration completed and you need to run the full crawl after adding the content sources.

References..
https://www.microsoft.com/en-us/download/details.aspx?id=30374
https://technet.microsoft.com/en-us/library/jj219738.aspx
https://blogs.technet.microsoft.com/praveenh/2013/02/06/create-a-new-search-service-application-in-sharepoint-2013-using-powershell/
https://blogs.technet.microsoft.com/meamcs/2014/09/04/configuring-sharepoint-2013-search-topology/
http://stevemannspath.blogspot.in/2013/06/sharepoint-2013-scaling-out-enterprise.html

Overview of search in SharePoint Server 2013

In this post we are going to discuss about SharePoint search architecture, SharePoint search components, SharePoint search databases, and the SharePoint search topology.

The search architecture contains:

1. Search components
2. Search databases

Overview of search components and search databases:

There are some major changes in SharePoint server 2013 when compared to SharePoint server 2010. Below are the 6 different search service components in SharePoint 2013.

• Crawl Component
• Content Processing Component
• Indexing Component
• Query Processing Component
• Analytics Processing Component
• Search Administration Component

Crawl Component:

1. This component takes care of crawling the content sources and collects the crawled properties and metadata and then passes crawled items to the content processing component.
2. The crawl component uses one or more crawl databases to temporarily store information about crawled items and to track crawl history.

Crawl database

1. The crawl database contains detailed tracking and historical information about crawled items.
2. This database holds information such as the last crawl time, the last crawl ID and the type of update during the last crawl.
3. Manages crawl operations.
4. Each crawl database can have one or more crawl components associated with it.

Content processing component:

1. This component receives the information (crawled items) from the crawl component and then processes and sends it to the indexing component. It also interacts with the analytics processing component and is responsible for mapping crawled properties to the managed properties.
2. The content processing component writes information about links and URLs to the link database.

Indexing Component:

1. This component receives the processed items from the content processing component and writes those items to the search index. Each Index files are stored on a disk in the server that hosts the index component.
2. It also receives the queries from Query processing component and sends back the results.

    Index partition:

• Dividing the search index into separate portions, called index partition.
• Each index partition holds one or more index replicas(mirror/copy) that contain the same information.

 Index partition vs index replicas:

An index replica is a copy of the index. This is commonly used for availability. For example, create a replica of the index on more than one server so that your queries can be served by more than one server.

A partition is a chunk of the index. Create a new partition for scale, the recommendation is 10M items. So if you are indexing a lot of content you may create multiple partitions.

Query Processing Component:

This component handles incoming query requests and sends them to the indexing component for results. It also takes care of query optimization.

1. Query Processing Component analyzes and processes search queries and results.
2. When the query processing component receives a query from the search front-end, it performs linguistic processing first (like word breaking and stemming), then analyzes and further processes the query to optimize precision, recall and relevance. In the end, the processed query will be submitted to the index component.
3. The index component returns a result set based on the processed query back to the query processing component.

Analytics Processing Component:

1. The analytics processing component performs two types of analyses: search analytics and usage analytics.
2. This component uses information from these analyses to improve search relevance, create search reports, and generate recommendations and deep links.
3. The results from the analyses are added to the items in the search index. In addition, results from usage analytics are stored in the analytics reporting database.

Search analytics VS usage analytics

Search analytics is about extracting information such as -- links, the number of times an item is clicked, anchor text, data related to people, and metadata – from the link database. This information is important to relevance.

Usage analytics is about analyzing usage log information received from the front-end via the event store. Usage analytics generates usage and statistics reports.

About the link database

The link database stores information extracted by the content processing component. In addition, it stores information about search clicks; the number of times people click on a search result from the search result page. This information is stored unprocessed, to be analyzed by the analytics processing component.

About the Analytics reporting database

The analytics reporting database stores the results of usage analytics. In addition, the analytics reporting database also stores statistics information from the analyses. SharePoint uses this information to create Excel reports that show different statistics.

About the event store:

1. The event store holds usage events that are captured on the front-end, such as the number of times an item is viewed.
2. These usage events are stored as log files on the application server that hosts the analytics processing component.

Search administration Component:

1. The search administration component is responsible for running a number of system processes that are essential to search.
2. This component manages administrative processes as well as changes to the search topology, such as adding or removing search components and servers.

Search administration database

Stores search configuration data, such as the topology, crawl rules, query rules, and the mappings between crawled and managed properties. Only one search administration database per Search service application.