Views relationships to source entities

@DamienMcKenna I think I have this working. You need to add the fields to the index; you do not need to add relationships to the view.

1. Create the search index.
2. Add your taxonomy fields to the index. In the selector, use the Add button for the top-level item, don't expand the sub-items. Use the data type Integer. If you also need to put the term names into the fulltext index or something, then you would add this field again, expand the sub-items, and use the Add button for the Name sub-item.
3. Create the view using your search index.
4. Add the filter. Choose "Vocab Name" as the filter, not any sub-items such as "Vocab Name > Taxonomy Term > Name"
5. You should now have the option to choose Dropdown or Autocomplete.

Screenshot of adding the term to the index:

Screenshot of which filter to add:

I also think that, if you have the NID, you could and should be able to relate to original node, and not to have to add all needed fields to the index. Is contra productive: why use more resursces when you already have the data in Drupal database?!
And from another perspective, you only care about texts in searches, why have to add images, for example, or other elements like buy it now form from commerce to index?
It could be implemented in Search API query plugin, quite like original views, if you have an relation, a secondary query could be made to Drupal's db on nids obtained from search index to relate all needed fields.
This way you could use the full potential of Drupal views on search results.

Here's the expected scenario:

• Create an index for content (nodes).
• Add a taxonomy term reference field to the index, e.g. "field_tags", as an integer field.
• Create a view of the index.
• Add a relationship to the view for the tags: "Taxonomy term referenced from Tags"
• Open the "Add filter criteria" dialog.
• Expected results: a list of all fields of the Tags vocabulary are shown alongside all of the content fields.
• Actual results: no fields from the Tags vocabulary are shown.

@damienmckenna: I think you don’t completely understand the way Search API works. In the Search API, you can only filter on (and sort by, search through, etc.) fields that you have explicitly added to the index. When you index just field_tags, then that integer entity ID is what’s indexed on the server, and only this can be used for filtering/sorting/…. If you want to be able to add filters for any of the taxonomy term fields, you will have to add them to the search index first, as @cboyden explained in #12.
The relationships in Views are purely used for the “Fields” of the view, they don’t influence the available filters or sorts at all.

I just ran into something similar as well, where the View seemed to be missing something ... And your comment @anybody makes total sense, Search API logic is just different :)

Here's my use case: I got libraries ("Library"), and they all have many images ("Library image"). The Library images are nodes themselves, referencing a single library.

I needed to create a search for fields from the Libraries, on values such as total cost, floor size, etc. I also wanted to show a single image of a library, so I had to somehow connect the library image with its library.

In a standard View I would add a Relationship, then add the reference field (a Library image referencing a Library) which is a field called field_library_ref, and then get this setting:

Configure Relationship: field_library_ref: Content
[ ] Require this relationship

It's the "Require this relationship" which is missing in a Search API view, so it's not possible to add a relationship.

But I figured out another method: When adding fields to a Search index, under the "Expand" option for the Reference field of the Library image, all the Library fields are available. So instead of defining the relationship in the View, they can be added to the Search index, and then the values get indexed, and are ready for use in the View.