Site Search > App Settings > Data Management > Crawler

The SearchStax Site Search solution offers a Crawler add-on for Enterprise clients. The Crawler indexes the pages of your website starting with a single root node. See Crawler Walkthrough for the full procedure.

Crawler Perspective

For perspective on the Crawler feature (and a video demonstration) see Data Ingestion for Site Search.

Contents:

This page covers the following topics:

• What is the Crawler?
• Configure the Crawler
• Crawler List
• Settings Tab
  • Crawler Name
  • Start URL
  • Crawl Depth
  • Schedule
  • Manage Fields for Search Index
  • Exclusions
• History Tab

What is the Crawler?

The SearchStax Site Search solution’s Crawler is an add-on connector that finds and indexes all of the pages of a website, making them searchable through a Search App.

The Crawler begins with a root URL and follows page links from there to all connected pages using the same corporate domain, subject to a configurable crawl-depth limitation.

Each Search App can have multiple crawlers, putting multiple websites into a single combined index.

The crawlers automatically refresh the index at a predetermined interval, updating, adding, and deleting index entries as needed.

Configure the Crawler

If the Crawler feature is enabled for your Enterprise account, you’ll find it listed under Site Search > App Settings > Data Management > Crawler in the Navigation Menu:

This link opens the Crawler list, which is initially empty.

Crawler List

A Search App can have one or more Crawlers, each indexing pages from one or more websites.

How Many Crawlers Can You Have?

An Enterprise account may be authorized to create several concurrent crawlers. This limit is applied to the account, not to individual Site Search Apps. The progress bar on this screen shows the number of crawlers and the account limit.

From this list, you can monitor crawler status, open an editor to create or modify a crawler, kick off an immediate crawl, or delete a crawler.

When you rerun a crawler, it updates records of existing pages and deletes the records of pages that are no longer reachable. When you delete a crawler, the web pages it found are removed from the index.

To initiate a crawl, check the crawler in the list and use the Crawl Now button.

To view a crawler’s details, settings, and history, click the desired crawler in the list.

Settings Tab

Clicking on a crawler in the Crawler List takes you to the Crawler Details screen. Select the Settings tab.

Crawler Name

Each crawler in your SearchStax account must have a unique name. The names can be multi-word, mixed-case, and alphanumeric. Site Search will ignore case when checking for redundant names.

Start URL

The crawler requires a starting or “seed” web page to anchor the crawling process. The crawl follows all the outgoing links from that page recursively until it runs out of pages that have the same DNS domain as the starting page. The crawler will not wander into other domains. Create a second crawler if you want to include pages from another domain in the same index. Your Search App can support more than one, subject to the terms of your contract.

The Start URL can also point to a sitemap file, such as:

https://example.com/sitemap.xml

or a sitemap-index file:

https://example.com/sitemap_index.xml

Crawl Depth

The “crawl depth” is the number of links crawled from the Start URL. It has three defaults, depending on the Start URL:

• If the starting page is a sitemap.xml file, the default crawl depth = 1.

• If the starting page is a sitemap-index.xml file, the default crawl depth = 2.

• Otherwise, the default crawl depth is “0” meaning “unlimited.”

You can manually select a crawl depth of 1 through 10 to customize your crawl.

Schedule

When enabled, the crawler repeats its crawl daily at the indicated local time. Subsequent crawls add newly found pages, delete pages that can no longer be found, and refresh the remainder.

Subject to your contract, Site Search will impose a limit of one crawl per day.

Manage Fields for Search Index

The crawler maps information about a web page to Solr schema fields in the Site Search index. Although the crawler has a default set of mappings, some customization is normal. The Fields table lets you edit and refine your field mappings.

For a discussion of the default field mappings, see Crawler Default Fields.

You can delete a field using the in-line trash-can icon in the rightmost column.

To add a new custom field, click the Add Custom Field button. This opens a field editing form.

Notes on the field options:

• Site Search will modify the Custom Field Name to indicate the field type and language. For instance, field “paragraph” will become “paragraph-txt-en” in the list of fields.

• Meta Tag Name retrieves the content of a named meta tag in the web page. The default field list includes the “description” and “keywords” meta tags.

• XPath uses an XPath formula to scrape the content of HTML tags in the page. For instance, “//p//text()” retrieves the content of all paragraph (p) elements.

• CSS lets us input a CSS class selector. The crawler will retrieve the content of all HTML elements that match the selector. For instance, “class~=name” will match any element whose class attribute contains “name” as a separate word within a space-separated list of words.

• System offers a droplist of internal Site Search fields about a web page, such as id, title, url, and document_type. Most of these are predefined default fields.

• Field Type is a droplist of Solr schema field types: Boolean, Date, Float, Integer, String, and Text. This has implications for how the data is indexed and queried. For instance, a “String” field requires an exact whole-string match, but a “Text” field is tokenized to index individual words.

• The Apply Transformations feature becomes available when defining string and text fields. This makes transformers available to normalize irregular field values during ingestion. See Crawler Transformers in the Help Center.

Exclusions

After your initial crawl, experience may show that the Crawler needs to be more limited in scope. Exclusions are rules that prevent the crawler from exploring every branch of your domain.

Exclusion URLs: Enter part or all of a URL (or regex pattern) as the basis of an exclusion rule. Site Search will interpret it according to one of the following contexts:

• Beginning with: Excludes any page with a URL that begins with this string.
• Contains: Excludes any page containing the indicated substring.
• Ending with: Excludes any page where the URL ends with this string.
• Matching regex: Excludes any page where the URL matches the indicated regular expression.

Additional controls:

• Plus (+) icon: Click here to add the exclusion to the list of active exclusions.
• Save Changes button: Click to persist the changes you have made on this screen.

To delete an exclusion, check the box on the left of the exclusion and click the trashcan icon.

History Tab

The History tab presents summary statistics of crawler runs.

Not all of the discovered links can be crawled successfully, usually because of inappropriate file types. The Items Indexed and URL Crawled columns give a general idea of how successful the crawl was.

Questions?

Do not hesitate to contact the SearchStax Support Desk.

The post Crawler appeared first on SearchStax Docs.

The SearchStax Site Search solution offers a Crawler add-on for Enterprise clients. The Crawler indexes the pages of your website starting with a single root node.

The crawler maps information about a web page to Solr schema fields in the Site Search index. This page presents reference information about the crawler’s default field mappings. See Crawler for instructions on adding custom fields.

Default Field Mappings

Upon creation of a new crawler, default fields are automatically set with the following definitions for a given document type.

• Mappings that cannot be changed are not displayed in the field mapping table.
• All other fields are displayed and mappings can be changed by the user by deleting the field and creating a new one.
• The following rich text formats are supported: .pdf, .docx, .xlsx, .pptx, .txt, .rtf.
• Note that the field labels visible in the crawler settings are shown in the Drupal, Sitecore, and Custom App columns of the table.

Questions?

Do not hesitate to contact the SearchStax Support Desk.

The post Default Field Map appeared first on SearchStax Docs.

This is an end-to-end walkthrough of indexing the SearchStax product documentation website using the SearchStax Site Search solution and its Crawler. This exercise takes about half an hour to complete.

Getting search results for your website is surprisingly easy, but there are moments when you wonder what to do next. This discussion captures those moments.

We assume that, as an Enterprise client, you have a Site Search account courtesy of your SearchStax Onboarding Manager. Log in to the Site Search interface.

The exercise begins with setting up and running the Crawler. When the crawl is complete, we walk though the Site Search features that create the first search experience for your site.

Contents:

This page covers the following topics:

• Create the Site Search App
• Configure and Run the Crawler
  • Configure the Crawler Fields
• Crawling
  • Inspect the Document Fields (Optional)
• Configure Search Fields
• Hosted Search Experience
• Configure Result Fields
• Configure a Facet
• Share Search Results
• Crawl Your Own Site?

Create the Site Search App

When you Create the Site Search App, be certain to configure it as a “Custom” application.

Configure and Run the Crawler

If the crawler feature is enabled for your account, you’ll find it listed under Site Search > App Settings > Data Management > Crawler in the Search App’s navigation menu:

This link opens the Crawler list, which is initially empty.

A Search App can have multiple crawlers, depending on the terms of your contract. Each crawler can index a different website. The list is initially empty. Click Create a Crawler.

The next step is to provide your crawler with a name and a starting URL. Site Search will verify that the URL is reachable.

The crawler begins with a root URL and follows page links from there to all connected pages within the same corporate domain, subject to a configurable crawl depth. (The optimum crawl depth for this example is 2, but the default value of 0 works, too.)

You can Crawl Now if you wish, but we advise you to visit the list of crawler fields first. The crawler is limited to one run per day, and we need to set up a special field before launching it.

Configure the Crawler Fields

This is an optional step to demonstrate setting up a facet. The crawler imports a set of default fields from webpages (see Default Field Map for details). You will find, however, that your target website uses additional fields. Site Search lets you add these fields to the crawl.

The SearchStax website doc pages contain a Products meta tag that makes a simple facet demonstration:

<meta name="Products" content="Managed Search">

We’d like the crawler to import the value of this tag to the index.

Open the Manage Fields for Search Index section of the crawler settings. You’ll see the list of default fields.

These fields could be useful in your project, and are harmless if not. Click the Add Custom Field button. The resulting dialog box is described on the Crawler page.

• Set the Custom Field Name to products. This is the label you will see in the Site Search lists of fields.
• Select the Meta Tag Name option and enter Products. This is the meta tag name from the target page HTML.
• For a field destined to become a facet, the string datatype is usually the best choice.
• We will not add any transformations at this time.

Click Add Field. The new field, labeled products_ss (your field name plus the string datatype), will appear in the list.

When you are satisfied with the setup, scroll to the bottom of the screen and click the Crawl Now button.

Crawling

As the crawl proceeds, you’ll see progress statistics updating.

Inspect the Document Fields (Optional)

This optional section presents some “tips and tricks” for inspecting the crawler’s output before configuring the Search Fields and Result Fields in Site Search. Although technical, the steps described here produce a convenient map of the indexed fields and their typical values.

Navigate to the Site Search > APP Settings > All APIs > Search & Indexing screen. The Read-Only authentication token is near the bottom of the screen.

You’ll need to copy the token to the clipboard and paste it into a text buffer temporarily.

Now, scroll back up the screen and find the App’s Select Endpoint.

Copy the endpoint to a text buffer and make these changes to it:

• Change emselect to select.
• Append ?q=*:*&wt=json&indent=true at the end following select.

Now we’ll assemble a Curl command in the text buffer. Use this format:

curl -H "Authorization: Token <Read-Only Token>" "<Select Endpoint>/select?q=*:*&wt=json&indent=true"

You should now have a URL similar to this:

curl -H "Authorization: Token 6e6a32<redacted numbers>597c5a" "https://searchcloud-1-us-west-2.searchstax.com/<redacted path>/select?q=*:*&wt=json&indent=true"

Paste this string into a Linux Bash command window (or a PowerShell terminal on Windows) and send it. It will return ten documents from your index, showing all the fields in use and their content. (Notice the products_ss custom field near the bottom of this list.)

      {
        "id":"https://demo.searchstax.com/docs/searchstudio/analytics-glossary/",
        "exif_tenant_id":"2",
        "exif_crawlid":"2151",
        "exif_crawl_definition_id":"43",
        "exif_appid":"studio-1810",
        "url":["https://demo.searchstax.com/docs/searchstudio/analytics-glossary/"],
        "paths":["docs / searchstudio / analytics-glossary"],
        "document_type":["html"],
        "date":"2024-06-24T02:36:12Z",
        "title":["Analytics Glossary - SearchStax Site Search Docs"],
        "headings1":["Analytics Glossary"],
        "headings2":["Questions?"],
        "description":["The SearchStax Site Search solution's Analytics Glossary is a summary of key terms and definitions used for analytics in Site Search."],
        "products_ss":["Site Search"],
        "content":["Analytics Glossary - SearchStax Site Search Docs Managed Search Site Search Help 

             <Most of the content was removed for clarity>

        ],
        "_version_":1802708265532915712}

If you have difficulty making this work, contact SearchStax Support for assistance.

This output will be a convenient resource in the following steps.

Configure Search Fields

At this point, the webpage data is in the index, but we can’t search it yet. Before we can search, we have to choose which fields to search.

Navigate to Site Search > Configurations > Basic Configurations > Search Fields. This screen tells Solr which index fields to search.

The left column contains the available fields in the schema (but they are not necessarily present in the crawled documents). Click on a field to move it into the list of searchable fields.

The title and description fields are pre-selected as a convenience to the user. Select the headings1, headings2, and headings3 fields, too. Headings are rich in relevant keywords. (The content field is pre-selected, but in this case, we suggest removing it from the search list.)

To experiment with a facet list, also add the product_ss field to this list. Facets must be based on search fields.

Then click Publish to re-issue the index. Publishing a small project like this one takes a couple of minutes.

Hosted Search Experience

In this exercise, we’ll use the Site Search Hosted Search Experience to view and test our search settings. This feature is a fully-functional search page supporting the many features controlled by Site Search. The Hosted Search Experience lets us configure a feature, publish it, and view the result a few seconds later.

You can do almost the same thing using the Site Search Preview screen, but that environment is tailored to engineers rather than website designers.

The Search Fields (above) are not always the best to display as Results Fields. We must now show Site Search how to present results in the Hosted Search Experience.

Navigate to Site Search > Configurations > Basic Configurations > Results Fields.

The Results Fields screen lets us select fields from the index to display in the Hosted Search Experience’s search results. Each field can be mapped to a preformatted position in a results “card.”

Choose a field from the Return Field list. If needed, add a human-friendly Label. Then, map the field value to a Results Card Field, as explained on the Results Configuration page. The (+) icon at the right adds the configured field to the list of display fields (the lower red box). Don’t overlook that step!

For this exercise, make the following mappings:

• Map the index’s URL field to the result card’s URL field. This is a default mapping. It makes the result items clickable and links them to the web pages they represent.
• Map the index’s title field to the result card’s Title field. This will put the page’s title at the top of the result summary. Again, this is a default setting.
• The index’s description field is mapped by default to the Description field of the result card.
• Map headings1, headings2, and headings3 to “No mapping” on the results card. This lists the field values below the result item’s description.
• Map the products_ss field to the result card’s ribbon field. This will display the product name as a banner above the result item.

At this point, a typical result item in the Hosted Search Experience looks much like this:

Configure a Facet

The search page seems incomplete without at least one facet list off to the side. How do we set that up?

Navigate to Site Search > Configurations > Basic Configurations > Faceting.

The Faceting page provides full instructions for operating this screen. To begin, check the box that enables faceting.

The Facet Fields list lets you select an index field for a facet. (If you don’t see the field in the list, click that Reload Schema button again.) Select products_ss. You can add a label to be the title of the facet list. In our example, the facet options will be ranked by count.

Click the (+) icon to add the facet to the Profile.

When finished, click the Publish button.

If you check the Hosted Search Experience at this point, you’ll find a new facet on display.

Share Search Results

Experience has taught us that a search project often has many more stakeholders than developers. The project will need a public search portal for stakeholders so they can contribute their insights and requirements without logging into Site Search.

In the navigation menu, navigate to Site Search > Configurations > Search UI > Hosted Search Experience.

This screen provides a URL to a shareable search environment. You can View the page immediately or use the Copy icon on the right to share the URL with coworkers.

Use the Regenerate button to refresh the Hosted Search Experience after making changes.

Crawl Your Own Site?

You can use this guide to perform an initial crawl of your own website. Watch for these issues:

• The time required for crawling scales up for the number of web pages and their size. The SearchStax site is about 1000 pages. Crawling time will vary from site to site.
• Crawler should retrieve all the fields of interest. Only some of them will be useful for searching. A few will make good facets. Others will be for display only.
• See Field Types for advice about setting up text and string facets using the Crawler. (The Crawler can index the same content into two index fields using different datatypes.)

Questions?

Do not hesitate to contact the SearchStax Support Desk.

The post Crawler Walkthrough appeared first on SearchStax Docs.

SearchStax provides a special module that connects Sitecore directly to the SearchStax Site Search solution and lets you publish a custom search page immediately.

For perspective on the advantages of the Sitecore Module, see Data Ingestion for Site Search in our blog space.

Contents:

• What is the Sitecore Module?
• Install the Sitecore Module
• Configure the Sitecore Module
• Populate Schema and Rebuild
• Reload the Search App’s Schema
• Create a Search Page
• Computed Fields

What is the Sitecore Module?

This package is a Sitecore module that Sitecore developers can install to leverage all the search capabilities offered by SearchStax Site Search for your client-facing search pages. It contains a Sitecore index connector which can index your Sitecore items using the out-of-the-box Indexing Manager provided by Sitecore. Not only that, but it comes with a functional search page that can be built in minutes instead of weeks of work. With this package your client can focus immediately on improving the real search experience, since that is where the business value is.

Install the Sitecore Module

The first step is to contact the SearchStax team and ensure you already have a SearchStax Site Search account and App created with a Solr core configured to deal with your Sitecore. Once you have this you are ready to go.

This is almost a plug-and-play module, thus installing it is as easy as possible.

Below are the supported Sitecore versions, with the links for downloading the Sitecore package:

Once you download the package, open the Sitecore Installation Manager:

Use the Choose File option to select the downloaded file and upload it to your Sitecore.

Once the package is uploaded it’s time to properly install it. You can do this by clicking on Next and then on Install.

The installation should take no more than a few minutes. You will know the installation is completed when you see the below pop-up. Go ahead and click on the Configure SearchStax for Sitecore Package button:

Configure the Sitecore Module

Navigate to the SearchStax Module Configuration Page

From the Sitecore Launch Pad, click on the Sitecore Module Configuration button under SearchStax Site Search

This will open up the Site Search for Sitecore configuration page.

Enter credentials and select account and app.

1. Start by entering your SearchStax Site Search username and password. Then press the Load Accounts button. This will populate the Site Search Account drop down.
2. Select the appropriate Site Search Account. It’s probable only one account is available, but you might have access to multiple accounts. Once selected, this will populate the available Search Apps within the selected Studio Account.
3. Select the appropriate Search App.
4. Once you have selected the above options, then press the Create an Index button.

Configure Site Search for Sitecore

The next page that the configuration will take you to is the Index Properties page for the Search App that you selected in the previous step.

In the case that your Search App is using Basic Authentication, go here. Otherwise, follow these directions:

1. Select your root Item(s). This is going to be the Sitecore Path where the Index should start it’s indexing from. You can list multiple root item paths if there are multiple locations within your Sitecore Content tree that content should be searchable from. By default, the module will index every page under /sitecore/content/home/.

2. (Optional) You are given an opportunity to enter the Hosted Search Page Password. At this point, unless you know what this password is, and you have previously configured it, we can safely skip this field.

3. Press the Save button. Now move on to the next step.

Configure Site Search for Sitecore with Basic Authentication:

• Select your root Item(s). This is going to be the Sitecore Path where the Index should start it’s indexing from. You can list multiple root item paths if there are multiple locations within your Sitecore Content tree that content should be searchable from. By default, the module will index every page under /sitecore/content/home/.
• Read Only Solr Password and Admin Solr Password: These are the “Read-Only API Credentials” and the “Read-Write Search API Credentials” from the Search and Indexing tab of the Site Search > App Settings > All APIs screen. You’ll have to scroll down to the bottom of the screen to find them. The User ID is set by the system, but you can set the password (both Read only and Read Write need to be set).

• (Optional) You are given an opportunity to enter the Hosted Search Page Password. At this point, unless you know what this password is, and you have previously configured it, you can safely skip this field.

• Reset Index When Rebuilding: Check this field if you want SearchStax to completely clear your index right before rebuilding it. Keep in mind that doing so your index will be empty during the rebuilding process, which might take minutes, and that your search page will return no results during this period of time.

• Press the Save button.

After pressing Save you’ll be prompted a confirmation box. Press confirm to allow Sitecore to apply the appropriate configuration files.

Validate Index Creation and Populate Managed Schema

Once you have installed and configured the Sitecore module, you must run a Populate Schema operation on this new Solr collection to update its schema with the one used by Sitecore. You can find more details about this at https://doc.sitecore.com/xp/en/developers/90/platform-administration-and-architecture/solr-managed-schemas.html. This is a common operation that must be executed on every Solr index used in Sitecore and is very easy to perform. Open your Sitecore admin page and click on Control Panel -> Populate Solr Managed Schema.

Select your previously configured core name and then click on Populate.

Once this operation is completed, you should see a success message like this:

Now go back to the Control Panel and click on Indexing Manager. When it opens, select your previously configure core name and click on Rebuild.

Usually, it should only take a few minutes but keep in mind that the time this operation will take is proportional to the number of items you have in your Sitecore. 

Once this operation is completed you should see a success message like this:

This means that your schema is updated and your SearchStax index is ready to receive new indexed items and search queries.

Reload the Search App’s Schema from within the Site Search Dashboard

At this point, we navigate away from Sitecore and login to your SearchStax Site Search Dashboard.

(You can search SearchStax Site Search directly by going to www.searchstudio.com or click on Search Management from within Sitecore as shown below.)

1. From within Site Search, select the App you are in the process of setting up.  Site Search > Profile Selector > Apps List.
2. Then navigate to Site Search > App Settings > All APIs, scroll to the bottom of the screen, and press the Reload Schema button.

Rebuild the Index

Head back to the Sitecore Control Panel and select Indexing Manager.

Select the appropriate index and then click the Rebuild button.

Create a Search Page

• Configure Result Fields in Site Search
  • Go to Site Search > Navigation Menu > Search App > Search > Results Configuration
  • Select all fields that you would like to be displayed in Result Items. You can map some of the fields to the Result Cards to show them in a special way. Make sure you have at least one field set up.
  • Click on Publish to publish the Results Configuration.

• Configure Search Fields in Site Search
  • Go to Site Search > Configurations > Basic Configurations > Search Fields.
  • From the Search Fields, select one or more fields that you would like to search on. At least one field should be selected.
  • Click on Publish.

The Sitecore Module comes with a plug-and-play search page template that you can use to create your search page in a few minutes. Use the /sitecore/templates/Feature/SearchStax/SearchPage/SearchStaxPage template to create it.

And then use the SearchStax Index field to select which index must be used by this search page when performing queries:

You just need to publish it to start testing your new search experience:

Computed Fields

The Sitecore Module uses a few computed fields out-of-the-box. These fields improve the search experience of your initial search page, but they can be replaced or removed as your project matures. The module’s computed fields are described in the following sections.

For general information on Sitecore’s computed fields, see https://doc.sitecore.com/xp/en/developers/91/platform-administration-and-architecture/create-a-computed-index-field.html.

ResultTitle

This field contains the title of the result item, based on the Sitecore item DisplayName property. If that is empty, the returned string is its Name property value.

This field is not required by the module to make it work properly. However, it is the field that the module uses as the result template title. If you decide to remove or change, do not forget to change the Site Search config connector.fields.title property on the /Views/SearchStaxPage/Index.cshtml.

ResultURL

This field contains the result URL. It leverages the native Sitecore MediaManager.GetMediaUrl method for media items to get the item URL. When the item is not a media item, this fields starts by defining the current website context. It uses the indexed item Paths.FullPath property and tries to find a website definition which has the same Database property value and has a RootPath which is a parent item from the current one. Once it finds the current website context, it then leverages the native Sitecore LinkManager.GetItemUrl method to get the item URL.

This field is not required by the module to make it work properly. However, it is the field that is used as the result template clickable URL. If you decide to remove it or change, do not forget to change the Site Search config connector.fields.url property on the /Views/SearchStaxPage/Index.cshtml.

RenderedContent

This field contains a string value that improves the search relevance of the website. It indexes not only the current item text fields, but also the text fields from every data source present on any given indexed page rendering, except the renderings on the <renderedContentFieldExcludedRenderings> configuration list.

In order to add new rendering ids to this list and remove them from being indexed by this computed field, please edit the < renderedContentFieldExcludedRenderings > node children’s values on the App_Config/Modules/SearchStax/SearchStax.Cloud.config file. This field is not required by the module to make it work properly. You can remove it from your index if you wish.

Adding a Custom Computed Field

What if you would like to add a custom computed field to the configuration? After the SearchStax index has been created, find the index within [site]/App_Config/Modules/SearchStax/SearchStax.Connector.Provider.config, where [site] is the folder of your inetpub/wwwroot/site. In the <fields hint=”raw:AddComputedIndexField”> section, add an entry for the CustomComputedIndexField:

<field fieldName="CustomComputedIndexFieldName" returnType="string">
     YourNameSpace.CustomComputedIndexField,YourNameSpace
</field>

Bear in mind that computed index fields are expensive and place a load on the CPU. One should minimize their use.

The post Sitecore Module 3.0.0 appeared first on SearchStax Docs.

How to upgrade from Sitecore Module v2.0.0 to v3.0.0

The SearchStax Site Search solution has released the latest version of its SearchStax for Sitecore package which contains several new features and bug fixes. If you are still using SearchStax for Sitecore 2.0.0 you should think about upgrading it in order to leverage all this. This document shows how you can upgrade from a previous version to 3.0.0.

Start by downloading SearchStax for Sitecore 3.0.0 version from here.

There are a few changes in the structure of the SearchStax configuration files between versions 2.0.0 and 3.0.0, so please move this file to a separate folder where it won’t be overwritten:

• \App_Config\Modules\SearchStax\SearchStax.Connector.Provider.config

Then delete all the config files in the following folder:

• \App_Config\Modules\SearchStax

Open your Sitecore instance and install the SearchStax for Sitecore 3.0.0 package using the Sitecore Installation Wizard.

Make sure you select “Yes to all” when asked to overwrite files:

Make sure you select “Apply to all” and “Overwrite” to overwrite all the items already in Sitecore.

Once the installation is finished, open the configuration page, and recreate the SearchStax indexes you need to be part of your Sitecore instance.

If you have custom attributes that were previously added to your indexes, such as computed fields, you can use the original 2.0.0 SearchStax.Connector.Provider.config file backed up at the beginning of this upgrade process to bring those changes to the new version of this file.

Keep in mind that we can’t simply overwrite the new file with the old one, since there are some structural changes on it as part of this new 3.0.0 release. Doing so would make your Sitecore instance not work properly.

Open both versions of the SearchStax.Connector.Provider.config  file (the backed up version and the new 3.0.0 version present on the \App_Config\Modules\SearchStax folder) and manually bring your customizations to the new one by using a text-editor program.

Example:

After doing so, open every search page using /sitecore/templates/Feature/SearchStax/SearchPage/SearchStaxPage template and use the SearchStax Index field to select which previously created index this given page should use when searching for results.

Rebuild all your SearchStax indexes using the Sitecore index manager.

Publish your website and you should now be able to use the search page without any issues.

DefaultSolrSearchIndexesInitializer: Contruction has failed.
Exception: SolrNet.Exceptions.InvalidURLException
Message: Invalid URL ''
Source: SolrNet
at SolrNet.Utils.UriValidator.ValidateHTTP(String s)
at SolrNet.Impl.SolrConnection..ctor(String serverURL)
at Sitecore.ContentSearch.SolrNetExtension.Impl.SolrConnectionEx..ctor(String serverUrl, Boolean usePostConnection)
at SearchStax.Feature.Connector.Provider.SearchStaxSearchIndex.GetSolrAdmin()
at SearchStax.Feature.Connector.Provider.SearchStaxSearchIndex.GetConnector()
at SearchStax.Feature.Connector.Provider.SearchStaxSearchIndex.InitializeSolr()
at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.InitializeIndexSystem()
at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.Initialize()
at Sitecore.ContentSearch.SolrProvider.Initialization.DefaultSolrSearchIndexesInitializer.ConstructIndexSafe(SolrInitializableIndex index)

Nested Exception

Exception: System.ArgumentNullException
Message: Value cannot be null.
Parameter name: uriString
Source: System
at System.Uri..ctor(String uriString)
at SolrNet.Utils.UriValidator.ValidateHTTP(String s)

The post Upgrade to Module 3.0.0 appeared first on SearchStax Docs.

Once your data is connected to the SearchStax Site Search solution, you will need to map fields from Sitecore to Site Search to see search results. This is accomplished in the Results and Display section where you can assign Sitecore Fields to Mapped Fields.

To simplify things, we created a Results Card with pre-built Mapped Fields. Out of the box, Site Search supports six mapped fields: Ribbon, Title, Paths, Date, Description, and URL. You can add Fields without a mapping as well and they will appended to the end of the Results Card.

Below is a table of common Sitecore Fields that map nicely to Site Search and are usually found in all Sitecore instances.

The post Mapping Sitecore Fields appeared first on SearchStax Docs.

SearchStax has compiled the following list of “frequent questions” about the SearchStax Site Search solution Sitecore Module.

Index rebuild operation is failing after a few seconds. How to fix it?

This may be happening due to multiple reasons. You can follow these tips to help you troubleshoot this issue:

1. Always check the Sitecore logs for more information. Sometimes the error message will point you to exactly where the error is.
2. Temporarily disable all the custom computed fields and try again. If it works, there is a issue that must be fixed in one of your computed fields.
3. Make sure that the crawler root item exists in the target database (usually web).
4. Confirm that there are no firewall rules blocking your Sitecore instance from sending requests to SearchStax cloud endpoints.

Index rebuild operation in Sitecore is taking too long. What can I do to make it faster?

Indexing items in Sitecore is a heavy operation by definition, so the first thing you can do is make sure your crawlers have the most specific root item possible. For example, instead of indexing /sitecore/content/site, change it to /sitecore/content/site/home, since all we want to index are pages.

Another thing that might help with indexing performance is limiting the number and complexity of computed fields. Remember, each computed field is executed one time for every item you want to index.

I want to have the page rendering data sources as searchable content. How can I index them?

First, you don’t need to index those items individually. The package comes with a computed field called RenderedContent that indexes them as part of the page itself. All you have to do is to configure it properly. You can find more information at https://demo.searchstax.com/docs/searchstudio/sitecore-module/#computed.

I have an exception message after adding a SXA rendering to my Sitecore page. How can I fix this?

SearchStax for Sitecore SXA search page requires a datasource item in order to work properly. Make sure you create it first and then add it in a proper placeholder. More information on https://demo.searchstax.com/docs/searchstudio/searchstax-studio-sitecore-sxa/

When doing A/B Testing with Sitecore, multiple versions of the page are being published and available with Search. How can I fix this?

When doing A/B testing, Sitecore publishes multiple versions of the page which are available to different users. To exclude multiple versions from the search results, you can set a Data Exclusion filter to get only the latest version of the page in the search result. This is captured by the field ‘_latestversion‘ and as can be set as shown below.

Simply add this to the Data Filters and publish the model to exclude the duplicates from search.

Can I use the Sitecore ContentSearch API with the index created by the Sitecore Connector for Site Search?

No, the index that is created by the connector is for indexing purposes only and will not work with the Content Search API.  Additionally, the index definition requires the master database, which is not available to Content Delivery servers.  Please utilize the Search API’s in order to query the SearchStax Site Search collection.

DefaultSolrSearchIndexesInitializer: Construction has failed.

DefaultSolrSearchIndexesInitializer: Construction has failed.
Exception: SolrNet.Exceptions.InvalidURLException
Message: Invalid URL ''
Source: SolrNet
at SolrNet.Utils.UriValidator.ValidateHTTP(String s)
at SolrNet.Impl.SolrConnection..ctor(String serverURL)
at Sitecore.ContentSearch.SolrNetExtension.Impl.SolrConnectionEx..ctor(String serverUrl, Boolean usePostConnection)
at SearchStax.Feature.Connector.Provider.SearchStaxSearchIndex.GetSolrAdmin()
at SearchStax.Feature.Connector.Provider.SearchStaxSearchIndex.GetConnector()
at SearchStax.Feature.Connector.Provider.SearchStaxSearchIndex.InitializeSolr()
at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.InitializeIndexSystem()
at Sitecore.ContentSearch.SolrProvider.SolrSearchIndex.Initialize()
at Sitecore.ContentSearch.SolrProvider.Initialization.DefaultSolrSearchIndexesInitializer.ConstructIndexSafe(SolrInitializableIndex index)

Nested Exception

Exception: System.ArgumentNullException
Message: Value cannot be null.
Parameter name: uriString
Source: System
at System.Uri..ctor(String uriString)
at SolrNet.Utils.UriValidator.ValidateHTTP(String s)

The post Sitecore Module FAQ appeared first on SearchStax Docs.

In a real-world project, customers using the SearchStax Site Search solution’s Sitecore Module to leverage all the Site Search capabilities will end up needing to customize the Sitecore search page in order to give it the same look and feel of the rest of the website, or to extend some missing feature. The best way to do that is by creating a custom search page.

Prerequisites

This article assumes that you already have a C# solution with an MVC project.

Assembly References

Start by adding project references to those two SearchStax assemblies. They are part of your SearchStax for Sitecore package:

• SearchStax.Foundation.Core.dll
• SearchStax.Feature.SearchPage.dll

Controller

Add the below class to your MVC project and give it a meaningful name following the MVC pattern, such as CustomSearchStaxController. This class will have an Index() action method that will be used by a Sitecore controller rendering later. This method loads everything required by a SearchStax view class to work properly and passes it to the view using the SearchPageModel object.

using SearchStax.Feature.SearchPage.Services;
using System.Web.Mvc;
using System.IO;
using System;
 
namespace SearchStax.Feature.SearchPage.Controllers
{
    public class CustomSearchStaxController : Controller
    {
        private readonly ISearchPageConfigurationFetcher _searchPageConfigurationFetcher;
 
        public CustomSearchStaxController()
            : this(new SearchPageConfigurationFetcher())
        { }

        public CustomSearchStaxController(ISearchPageConfigurationFetcher searchPageConfigurationFetcher)
        {
            this._searchPageConfigurationFetcher = searchPageConfigurationFetcher;
        }
 
        public ActionResult Index()
        {
            var searchModel = this._searchPageConfigurationFetcher.GetSearchPageConfiguration();
 
            string viewName = "Index";
            var viewFilePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Views\\SearchStaxPage\\" + searchModel.SearchStaxConfiguration.IndexCoreName + ".cshtml");
            if (System.IO.File.Exists(viewFilePath))
            {
                viewName = searchModel.SearchStaxConfiguration.IndexCoreName;
            }
 
            return View(viewName, searchModel);
        }
    }
}

View

Create a new view file based on the already existing \Views\SearchStaxPage\Index.cshtml file found on your Sitecore instance. Name this file as Index.cshtml and add it to the project folder \Views\[CustomSearchStax]. Make sure you change the [CustomSearchStax] by your controller prefix.

This is the moment when you can apply some custom styling with CSS manipulations, or even more structural changes to the search page, accordingly to what is required by your project design.

Rendering

In Sitecore, create a controller rendering under a custom folder on /sitecore/layout/Renderings. Configure the Controller field to point to the previously created controller, and the Controller Action field to its Index method. Here is an example:

Search Page

Once you have everything configured in the rendering, you can add the new rendering to a page item and validate that the customizations you previously applied are there.

You can even compare this search page with the standard one that uses the standard /sitecore/layout/Renderings/Feature/SearchStax/SearchPage/SearchStaxPageRendering to see the customizations:

Summary

1. Chose an MVC project or create a new one.
   1. Add a reference to SearchStax.Foundation.Core.dll
   2. Add a reference to SearchStax.Feature.SearchPage.dll
2. Create a new controller called CustomSearchStaxController
   1. Copy and paste the controller class from the example
3. Create a new view based on \Views\SearchStaxPage\Index.cshtml and add it to \Views\ CustomSearchStax\Index.cshtml
4. Create a new rendering pointing to the method in the new controller
5. Add the rendering to a search page and validate the customizations

The post Custom Search Page appeared first on SearchStax Docs.

The Sitecore Module connects your Sitecore CMS to the SearchStax Site Search solution for a customized search experience. The Module is helpful and convenient up to the moment when you decide to revise one or more settings. At that point, you’ll need to edit a configuration file.

A typical scenario occurs when you decide to connect your Sitecore instance to a different Search App. This entails changes to Solr endpoints as well as to the user names and passwords of your Admin and Read-Only Solr users. Spelling correction or auto-suggest endpoints might or might not be included at your preference.

The file to edit is \App_Config\Modules\SearchStax:

<configuration xmlns:patch=http://www.sitecore.net/xmlconfig/>
  <sitecore>
    <searchstax>
      <configs>
        <cloud type="SearchStax.Foundation.Core.Configuration.SearchStaxConfiguration, SearchStax.Foundation.Core" coreIndexName="searchstudio-for-sitecore-test">
          <RootItem>/sitecore/content/tenant/site/home</RootItem>
          <SearchEndpoint>https://ss123456-bhfq2ixl-us-east-1-aws.searchstax.com/solr/searchstudio-for-sitecore-test/emselect</SearchEndpoint>
          <SuggestEndpoint>https://ss123456-bhfq2ixl-us-east-1-aws.searchstax.com/solr/searchstudio-for-sitecore-test/emsuggest</SuggestEndpoint>
          <AuthenticationBasicToken>YXBwNzAtYXBpOkFycmliYSEyMw==</AuthenticationBasicToken>
          <AdminAuthenticationBasicToken>YXBwNzAtYWRtaW46S29uYWJvcyEyMw==</AdminAuthenticationBasicToken>
          <SolrUsername>app70-api</SolrUsername>
          <SolrPassword>Arriba!23</SolrPassword>
          <AdminSolrUsername>app70-admin</AdminSolrUsername>
          <AdminSolrPassword>Arriba!23 </AdminSolrPassword>
          <AnalyticsApiKey>cBVKGLpV3bhcv8vu PBdP0hC5FtxcGnhwOvkK5h6Berw</AnalyticsApiKey>
          <IndexCoreName>searchstudio-for-sitecore-test</IndexCoreName>
        </cloud>
      </configs>
    </searchstax>
  </sitecore>
</configuration>

The meaning of most of these settings should be self-evident from the parameters required by the Sitecore Module. The exceptions would be the AuthenticationBasicToken and the AdminAuthenticationBasicToken.

• AuthenticationBasicToken: The authentication token for the Read-Only Solr user.
• AdminAuthenticationBasicToken: The authentication token for the Admin Solr user.

You did not have to provide these values when setting up the Sitecore Module, but any changes to the Read-Only or Admin user credentials will force you to generate new tokens and enter them here along with the new credentials.

Fortunately, there are web sites that generate authentication tokens based on username and password combinations. One of them is the Basic Authentication Header Generator:

Just copy the token from the screen and paste it into the config file.

After editing and saving the config file, you’ll have to restart Sitecore to see the changes take effect.

The post Modify Configs appeared first on SearchStax Docs.

In this guide you will learn how to create a field to facet or sort on in the SearchStax Site Search solution.

You may have already tried to create a facet or sort in Site Search, but didn’t get the results you were expecting. This is because fields of type text_general are tokenized. The data in them are broken up into tokens making any Faceting/Sorting behave unpredictably.

As an example, if you want to facet on the field city_t, you would expect the value “Los Angeles” to appear as a facet value. However, since city_t is a text_general type, “Los Angeles” actually becomes “Los” and “Angeles”. Your facet now has values for “Los” and “Angeles” which is not the expectation.

In this example, the solution is to change city_t to a “string type” field. String types are not tokenized. The data remains as in. So, the facet value would be “Los Angeles” as expected.

Now, let’s update your text_general field to a string type in Sitecore.

Change Field Type in Sitecore

1. Begin by creating a new XML file named Patch.config in the app_config/include directory. Paste the provided XML configuration into this file. The configuration will modify the return types of specific field types in Sitecore. This is how the configuration looks:
   

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
    <sitecore>
        <contentSearch search:require="solr">
            <indexConfigurations>
                <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider">
                    <fieldMap type="Sitecore.ContentSearch.SolrProvider.SolrFieldMap, Sitecore.ContentSearch.SolrProvider">
                        <fieldTypes hint="raw:AddFieldByFieldTypeName">
                            <fieldType fieldTypeName="html|rich text|single-line text|multi-line text|text|memo|image|reference" returnType="text">
                                <patch:attribute name="fieldTypeName">html|rich text|multi-line text|text|memo|image</patch:attribute>
                            </fieldType>
                            <fieldType fieldTypeName="single-line text|reference" returnType="string"/>
                        </fieldTypes>
                    </fieldMap>
                </defaultSolrIndexConfiguration>
            </indexConfigurations>
        </contentSearch>
    </sitecore>
</configuration>

Update the Schema

1. In the Sitecore desktop interface, select the Sitecore button at the bottom left (or the Sitecore logo at the top left if you’re using the newer interface).
2. Select Control Panel.
3. In the Control Panel, under the Indexing section, select Populate Solr Managed Schema.

Reindex the Data

1. In the Sitecore Content Editor, go to the Control Panel.
2. Under the Indexing section, select Index Manager.
3. In the Index Manager screen, select the relevant indexes and click on Rebuild.

Create a Facet/Sort in Site Search

1. Login to the Site Search dashboard at searchstudio.searchstax.com.
2. Navigate to Settings > Search API. At the bottom of the page click Reload Schema.

The post Field for Faceting/Sorting in Sitecore appeared first on SearchStax Docs.