githaven-fork/docs/content/doc/advanced/repo-indexer.en-us.md
guillep2k 72f6d5c882 Restrict repository indexing by glob match (#7767)
* Restrict repository indexing by file extension

* Use REPO_EXTENSIONS_LIST_INCLUDE instead of REPO_EXTENSIONS_LIST_EXCLUDE and have a more flexible extension pattern

* Corrected to pass lint gosimple

* Add wildcard support to REPO_INDEXER_EXTENSIONS

* This reverts commit 72a650c8e42f4abf59d5df7cd5dc27b451494cc6.

* Add wildcard support to REPO_INDEXER_EXTENSIONS (no make vendor)

* Simplify isIndexable() for better clarity

* Add gobwas/glob to vendors

* manually set appengine new release

* Implement better REPO_INDEXER_INCLUDE and REPO_INDEXER_EXCLUDE

* Add unit and integration tests

* Update app.ini.sample and reword config-cheat-sheet

* Add doc page and correct app.ini.sample

* Some polish on the doc

* Simplify code as suggested by @lafriks
2019-09-11 20:26:28 +03:00

2.4 KiB

date title slug weight toc draft menu
2019-09-06T01:35:00-03:00 Repository indexer repo-indexer 45 true false
sidebar
parent name weight identifier
advanced Repository indexer 45 repo-indexer

Repository indexer

Setting up the repository indexer

Gitea can search through the files of the repositories by enabling this function in your app.ini:

[indexer]
; ...
REPO_INDEXER_ENABLED = true
REPO_INDEXER_PATH = indexers/repos.bleve
UPDATE_BUFFER_LEN = 20
MAX_FILE_SIZE = 1048576
REPO_INDEXER_INCLUDE =
REPO_INDEXER_EXCLUDE = resources/bin/**

Please bear in mind that indexing the contents can consume a lot of system resources, especially when the index is created for the first time or globally updated (e.g. after upgrading Gitea).

Choosing the files for indexing by size

The MAX_FILE_SIZE option will make the indexer skip all files larger than the specified value.

Choosing the files for indexing by path

Gitea applies glob pattern matching from the gobwas/glob library to choose which files will be included in the index.

Limiting the list of files prevents the indexes from becoming polluted with derived or irrelevant files (e.g. lss, sym, map, etc.), so the search results are more relevant. It can also help reduce the index size.

REPO_INDEXER_INCLUDE (default: empty) is a comma separated list of glob patterns to include in the index. An empty list means "include all files". REPO_INDEXER_EXCLUDE (default: empty) is a comma separated list of glob patterns to exclude from the index. Files that match this list will not be indexed. REPO_INDEXER_EXCLUDE takes precedence over REPO_INDEXER_INCLUDE.

Pattern matching works as follows:

  • To match all files with a .txt extension no matter what directory, use **.txt.
  • To match all files with a .txt extension only at the root level of the repository, use *.txt.
  • To match all files inside resources/bin and below, use resources/bin/**.
  • To match all files immediately inside resources/bin, use resources/bin/*.
  • To match all files named Makefile, use **Makefile.
  • Matching a directory has no effect; the pattern resources/bin will not include/exclude files inside that directory; resources/bin/** will.
  • All files and patterns are normalized to lower case, so **Makefile, **makefile and **MAKEFILE are equivalent.