Specification document for Advanced search in Starshot

First version of document was created: https://docs.google.com/document/d/1GzrWO2gLUAIeWDHBGOuXnbOpAdM9q-WZGRnD...

I have reviewed the document and left my suggestions, please check.

The specification and scope document needs review.

Please add your remarks and suggestions as comments in this issue.

You can also have comment access to the Google document, please request it in a comment.

I like everything I read. I can think of plenty of ways that it can be extended but it makes a lot of sense to start simple. I would like to understand the JavaScript front end in detail at some point, but since you're providing a standard front end display that sounds as though it can only be a big bonus.

My main feedback is I'm not 100% on the content type facet. To me, it makes sense for say Event and Blog post, but if we end up with a Landing page and/or Basic page, those would seem strange as facets. Feels like a bit of a Drupalism to expose these to the end user.

What do others think?

Yeah agreed with #7, also people will be creating their own content types and/or installing other recipes which may or may not make sense as a facet. Also if that's the only facet included, dropping it would mean facets module itself isn't required as a dependency - for me I haven't used facets module much, but I've seen via the core queue a lot of issues between facets and views AJAX support, e.g. 🌱 [META] Overview of Facets + Ajax issues Active .

Thank you for the valuable comments and especially for the questions and doubts provided in #7 and #8.

At the moment this is just a proposal, we will see how it works, once we have created the recipes, and then they might change.

For us it would be ok, if we agreed that this is what we want to have in Starshot. If it turns out that some things cannot be included, because they are not stable enough or are incomplete, then they will not be removed from the corresponding recipes.

Regarding the factet for content type: @pameeela - you are right, there are scenarios in which a content type for facet does not make sense. But given, that Starshot does not have a data model yet (for example with a taxonomy that exists on all content types), the bundle is the only facet source we could think of. We think providing a facet makes sense, so the target personas can review, change, or build their own facets based on the example.

Regarding the issues for facets and AJAX: @catch - this is true, with the current version it is only possible to create a good solution with AJAX if you make many modifications and use patches (we have suffered from this in many projects). However, new versions are being released and there are two branches. We are in contact with @borisson_, one of the maintainers, and we think that we can provide a version for Startshot that will work nicely. The worst that could happen, is that we dismiss facet functionality with views.

#8 I think as we create more recipes over time facets it is likely to be included for other things, an example use case is for filtering documents. Just now it's early days so we don't have a roadmap but I think eventually we will have different flavours and if your use case doesn't require any filters, or it doesn't even require search, then you won't have that feature and you won't have its dependencies either.

Drupal CMS is the official name for Starshot → and therefore I changed the title of this issue and also replaced all mentioning to Starshot in our document.

I went through the document and it made sense for a simple start as mentioned in #6

However, here is my experience working with Search functionalities for client projects. Most of the time, clients require tracking of the search keywords and search results. Search API Tracking → has been the go-to module for this with some modifications sometimes based on the requirement, but the module does provide good foundational tracking. This can be used in conjunction with the Dashboard → recipe by providing a widget.

Most of the time, clients require tracking of the search keywords and search results.

I'd be curious to know how this squares with the Privacy track.

@phenaproxima that shouldn't be an issue, if that data is not processed to create user profiles, i.e. behaviour patterns and the likes. Of course, it depends: is the data stored externally, is the search linked to a specific user, or is the tracking used to improve the content only, etc.

But I'll take a note to cover that topic in the privacy track as well, at lest as an educational item.

I like the initial setup but have 2 remarks / questions that I haven't seen pop up here yet:

1. Have you considered indexing a full view mode? We have noticed that this became somewhat required when we started using layout builder, as that included content that wasn't easily transferable to a index view mode. This did however cause some issues with indexing certain non-page-specific content (e.g. a "related items" block).
2. Do we want to set something up to allow excluding certain pages from the index by default? We tend to include the search_api_exclude module in pretty much all of our projects (especially for 404/403 pages and the homepage).

1. The base Drupal CMS recipe sets up Drupal core search. How are your recipes going to extend or unset those things?

2. Consider adding the Search 404 module → as a different pathway for getting users into search. I feel like it belongs here more than the SEO recipe.

But given, that Starshot does not have a data model yet (for example with a taxonomy that exists on all content types), the bundle is the only facet source we could think of. We think providing a facet makes sense, so the target personas can review, change, or build their own facets based on the example.

I don't really agree with this approach, I think we should only include facets if it is a useful feature in the context we are providing it. It's true that we don't have a comprehensive content model, so we should wait for that, and if it (or something else) ends up as a good use case facets then that would be a reason to add it. Looks like this is pretty much what Tony said in #10 too.

The base Drupal CMS recipe sets up Drupal core search. How are your recipes going to extend or unset those things?

That seems like a bug if this recipe is going to exist to me - is there an issue for that?

I would wait on creating individual issues for undoing things in the repo, because it's just a copy of the prototype. We should/will make a plan for transitioning properly rather than doing it ad hoc.

I think it would have been better to start with a clean repo and bring any individual pieces in one by one, but maybe that can happen in a new branch.

That seems like a major issue to me, Drupal CMS shouldn't be providing two different search recipes that compete with each other. It will be a mess if someone starts with core search then enables this later.

We can just...remove the core search recipe from Drupal CMS in favor of our new one. Easy-peasy. No need to undo anything, and nothing will compete - recipes should never need to undo stuff. There are already several recipes, from the prototype, that are variations on core recipes. It's a familiar pattern. Maybe I'm misunderstanding, but I see no reason for concern here.

Huh? This isn't a major issue. We can just...remove the core search recipe from Drupal CMS in favor of our new one.

It would be a major issue if Drupal CMS shipped a 'simple search' recipe based on core search, which was then modified by an additional 'advanced search' recipe based on search_api, because then the advanced search recipe would have to manage the conflicts as @thejimbirch points out.

But it sounds like the simple search recipe is from the prototype, not intended to end up in a base Drupal CMS recipe at all, which is fine, but not the implication of #17.

Sounds like it hasn't been determined if the core search will be included or not, and whether this advanced search will be the search, which is fine. A third option would be to present both options as add-ons.

With my experience coming from an agency world, using search as the example, we could group search into 4 different solutions.

No search - Some site do no have site wide search. This is rare for us, but does happen.
Basic Drupal search - lower budget, you get what you get OOTB
Search API Database Search - Let's build an MVP now and make it better post launch in ongoing support.
Search API Solr/Faceted/configured/etc - For planned search implementations

For Drupal CMS, I don't think #1 is an option which is fine. The base recipe has #2. I am not as technical as you all, but I don't see moving from #2 to #3 is a big lift as Search API can extend Search, and we have non-destructive config actions such as disable and hideComponent(s). It would be similar to a site builder that started with the Standard profile and wanted to add on and configure Search API.

I believe this recipe proposal sets up #3 to #4 nicely where a developer or site builder would have to manually configure the next steps.

Core search will not be included. We will use the search capability from this recipe.

Whether that recipe is applied by default in part or in full is TBD.

Also once this is in Drupal CMS and once Drupal CMS has a stable release, I also think we should move Drupal core search to contrib. search_api is installed on more than 100,000 Drupal sites. Core search is also installed on a lot of sites but we can be pretty sure that's because it's in the standard profile and that isn't really going to be a thing any more once this all gets going.

Just FYI - core search was removed from the Drupal CMS repository in fb99d606e. Hopefully that clears up a little confusion.

Regarding #16 (view mode, search_api_exclude)

Yes, the view mode Search index will be indexed, compare

Note: That the view modes Search index and Search result highlighting input must have sensible fields visible. Changing this configuration will not be part of the recipe. However, the recipe will activate the view modes.

We could add search_api_exclude, which we also use in many projects, but it is a bit of a hidden feature and un-experienced users can easily miss it. My feeling tells me, that we should leave it out for the moment. But it could be added easily.

Regarding facet for content type #18 🌱 Specification document for Advanced search in Drupal CMS Needs review

We can take it out or make it a seperate recipe any time. Let's create it and take it from there.

Regarding removing core search #27

Thank you :-)

Seems like the search backend from this recipe needs to be applied by default, so that other recipes providing content types can configure the search index view mode for their content type.

The plan is solid and the technical side looks awesome, it's almost exactly what I'd hope for as a developer. I'm basing my coments mainly on the survey results blog post: https://www.1xinternet.de/en/highlights/community-survey-results-drupal-...

The only concerns I have are naming and the amount of recipes.

For naming, using Search API Base Recipe, or Facet recipe doesn't feel like it's aimed at the ambitious sitebuilder, who is the main persona to take into account for Drupal CMS. We should ensure the names and descriptions ensure people with no or very limited knowledge of Drupal will know this is exactly what they'd need. So using Search API, Views, Facets might be confusing. Potential names for recipes that resonate more for me taking the target persona into account, are e.g. Basic search, Search filtering, Search autocomplete.

This relates also to the split. I feel it should be possible to install one recipe to have search being active on your site, now if I understand correctly, you need at least the base recipe and the views recipe.

Curious to hear other people's thoughts on this.

@daften thank you for your feedback. Indeed it is very important to name the recipe properly and have a description that will help users to find the correct project for their needs.

There will be no need for end user to install several recipes, as recipes can depend on other recipes.

The reason that facets recipe will be separate is only that it brings new module dependency that maybe not everyone will need. Anyway if the recipe "Faceted search" (or any other name it will be using) will be installed it will of course install everything that is needed, including the "Base search".

The same reason "Autocomplete" will also be a separate recipe, as it has another module dependency, that should be optional.

See as example how "Events" recipe is done https://www.drupal.org/project/events_registration → and how it has the extensions for Calendar https://www.drupal.org/project/events_calendar → , Locations https://www.drupal.org/project/events_locations → and registration https://www.drupal.org/project/events_registration → . So the same idea will be for Search recipe, then there will be extension for Faceted filters and Autocomplete.

This work was finished. All results were incorporated and validated in the survey. The recipe will be created accordingly.

Automatically closed - issue fixed for 2 weeks with no activity.