Query blocks have implicit names, such as `select#1`, formulated
by appending their select number to the string `select#`. This patch
allows hints to scope their applicability by implicit query block
name. For example,
SELECT /*+ JOIN_ORDER(@`select#2` t1, t2) */ ...
@`select#2` is an implicit query block name. Users can control hint
applicability per query block without first naming the blocks with
QB_NAME().
Hints may now be specified within VIEWs during their creation and
they are applied locally within that VIEW's scope. For example,
CREATE VIEW v1 AS
SELECT /*+ IGNORE_INDEX(t1 idx1) */ FROM t1 ... GROUP BY ... HAVING ...
In many cases and for some parts of the VIEW, the query plan
doesn't really depend on how the VIEW is used, so it makes sense
to control a part of the query plan from the VIEW definition.
Implicit names are not yet supported in VIEWs. Attempting to create
a VIEW with an implicit name reference will cause the server to create
the VIEW, but it will emit a warning and exclude that hint from the query.
Allows [NO_]MERGE(@qb_name) and [NO_]MERGE() to apply to all tables
in a target named query block and all tables in the current query
block, respectively.
Repairs a bug when fixing hints for derived tables that prevented
derived tables from being linked with a query block's hints in the
case that no derived tables were explicitly specified in the query.
Introduces NO_INDEX_MERGE and INDEX_MERGE, which control whether or
not index merge strategies are used during query optimization. Here
is an example query from the tests:
SET optimizer_switch='index_merge_intersection=off';
EXPLAIN SELECT /*+ INDEX_MERGE(t1 f4, f2) */ COUNT(*) FROM t1
WHERE f4 = 'h' AND f2 = 2;
with the hint in place, the query plan will employ the index_merge
intersect strategy (abbreviated EXPLAIN output):
type Extra
index_merge Using intersect(f2,f4); Using where; Using index
The presence of the [NO_]INDEX_MERGE hint overrides the optimizer's
choice of keys during the index merge optimization. As we see in
the above example, keys f2 and f4 and given and the optimizer will
consider only those keys for this query.
When the hint is given without any particular keys, as in
INDEX_MERGE(table), then all keys are considered. In this case, the
cheapest index merge among the keys should be used. When
NO_INDEX_MERGE(table) is given, then index merge is disabled for
that table.
When the hint is given with one or more keys, then only those keys
are considered. In the case of NO_INDEX_MERGE, those keys are
excluded. This can lead to no merged indexes at all, because
there may not be sufficient row-ordered read columns available for
consideration.
The index merge strategies of intersection, union, and sort union
cannot themselves be directly controlled via the hints. In combination
with the optimizer switches for the same, the strategy may be
indirectly controlled but this is not guaranteed.
When the hint directs the optimizer such that insufficient ROR scans
are available, thus leading to a situation where the INDEX_MERGE hint
cannot be honored, the server will emit a warning to that effect.
In the hints module (opt_hints*{cc,h}), this commit adds some
index merge-specific functionality to make interpreting hint state
at callsites in the optimizer cleaner and more intuitive.
Particularly, we add a bit field to the table hints class which
indicates the keys that are marked by an [NO_]INDEX_MERGE hint, if
present. A new function, index_merge_hint (and associated new
helper functions) relies on this field when interpreting index merge
hint state for the optimizer.
If there are no index merges available prior to attemping to find
a suitable union/sort union, then the optimizer will not attempt
it. This change results in optimizer trace output which does not
include the 'analyzing_index_merge_union' block when there are no
merges.
Parts of this implementation based on MySQL commit
ebcb981807e3d91a64782e89d48e1a25622eafea
Introduces code cleanup and small refactorings, such as:
- Flip keys_to_use.is_clear_all() and de-dent
- Rename best_trp to best_read_plan
- Fix key_parts assignment spacing
- find_shortest_key now keeps happy path to the left
- Cleanup in the get_best_ror_intersect, remove_nonrange_trees,
and restore_nonrange_trees functions
- Reorganization of locals within test_quick_select
- De-dent some large code blocks by changing
if (key)
{
LOTS-OF-CODE;
}
into
if (!key)
continue;
LOTS-OF-CODE;
This commit introduces two optimizer hints: ROWID_FILTER and NO_ROWID_FILTER.
They allow explicit control over whether certain indexes are used to build
rowid filters.
Syntax:
[NO_]ROWID_FILTER(tbl_name@query_block_name [index_name [, index_name] ...])
Hints can be applied at two levels:
Table-level (only table name specified): enables or disables rowid filters
for the given table.
Index-level (table and index names specified): forces the server to use
or ignore the listed indexes when building rowid filters.
Examples:
SELECT /*+ rowid_filter(t1)*/ a FROM t1 WHERE a < 'e' AND b > 't';
SELECT /*+ no_rowid_filter(t1)*/ a FROM t1 WHERE a < 'e' AND b > 't';
SELECT /*+ rowid_filter(t1 key_a, key_b)*/ a FROM t1 WHERE a < 'e' AND b > 't';
SELECT /*+ no_rowid_filter(t1 key_c)*/ a FROM t1 WHERE a < 'e' AND b > 't';
===== Interaction with other index hints =====
If [NO_]INDEX, [NO_]JOIN_INDEX hints are specified for the same table,
then only indexes whitelisted by INDEX() or JOIN_INDEX() can be considered
for rowid filters. ROWID_FILTER() hint cannot force the use of indexes that are
disabled by NO_INDEX() / NO_JOIN_INDEX() or omitted from INDEX() / JOIN_INDEX().
For example, if one index should be used for data access and another for
a rowid filter, the filter index must be mentioned in both hints:
SELECT /*+ index(t1 key_a, key_b) rowid_filter(t1 key_b) a FROM t1 ....
When a hint has a list of index names, for example,
`NO_INDEX(t1 idx1, idx2)`
there is a possibility that some or all of the listed index names will
not be resolved. If none of them are resolved, the hint becomes a table-level
hint, for example, `NO_INDEX(t1)`, which erroneously disables all indexes
of `t1` instead of disabling only some of them.
This commit addresses this issue by adding an additional check: a hint
containing a list of index names is considered resolved only when at least one
of the listed names is resolved successfully.
Part of an umbrella task MDEV-33281 for implementing optimizer hints.
This commit introduces hints affecting the use of indexes:
- JOIN_INDEX, NO_JOIN_INDEX
- GROUP_INDEX, NO_GROUP_INDEX
- ORDER_INDEX, NO_ORDER_INDEX
- INDEX, NO_INDEX
Syntax of index hints:
hint_name([@query_block_name] tbl_name [index_name [, index_name] ...])
hint_name(tbl_name@query_block_name [index_name [, index_name] ...])
JOIN_INDEX, NO_JOIN_INDEX: Forces the server to use or ignore the specified
index or indexes for any access method, such as ref, range, index_merge,
and so on. Equivalent to FORCE INDEX FOR JOIN, IGNORE INDEX FOR JOIN.
GROUP_INDEX, NO_GROUP_INDEX: Enable or disable the specified index or indexes
for index scans for GROUP BY operations. Equivalent to the index hints
FORCE INDEX FOR GROUP BY, IGNORE INDEX FOR GROUP BY.
ORDER_INDEX, NO_ORDER_INDEX: Causes the server to use or to ignore
the specified index or indexes for sorting rows. Equivalent to
FORCE INDEX FOR ORDER BY, IGNORE INDEX FOR ORDER BY.
INDEX, NO_INDEX: Acts as the combination of JOIN_INDEX, GROUP_INDEX
and ORDER_INDEX, forcing the server to use the specified index or indexes
for any and all scopes, or as the combination of NO_JOIN_INDEX, NO_GROUP_INDEX
and NO_ORDER_INDEX, which causes the server to ignore the specified index
or indexes for any and all scopes. Equivalent to FORCE INDEX, IGNORE INDEX.
Two kinds of index hints were introduced during implementation:
the global kind for [NO_]INDEX hint, and the non-global kind for all others.
Possible conflicts which will generate warnings:
- for a table level hint
- a hint of the same type or the opposite kind has already been specified
for the same table
- for a index level hint
- the same type of hint has already been specified for the same
table or for the same index, OR
- the opposite kind of hint has already been specified for the
same index
- For a multi index hint like JOIN_INDEX(t1 i1, i2, i3), it conflicts
with a previous hint if any of the JOIN_INDEX(t1 i1), JOIN_INDEX(t1 i2),
JOIN_INDEX(t1 i3) conflicts with a previous hint
Support for optimizer hints NO_SPLIT_MATERIALIZED and
SPLIT_MATERIALIZED. These hints allow fine-grained control
of the "lateral derived" optimization within a query.
Introduces new overload of hint_table_state function which
tells both a hint's value as well as whether it is present.
This is useful to disambiguate cases that the other version
of hint_table_state cannot, such as when a hint is forcing
a behavior in the optimizer that it would not normally do
and the corresponding optimizer switch is enabled.
Implements and tests the optimizer hints DERIVED_CONDITION_PUSHDOWN
and NO_DERIVED_CONDITION_PUSHDOWN, table-level hints to enable and
disable, respectively, the condition pushdown for derived tables
which is typically controlled by the condition_pushdown_for_derived
optimizer switch.
Implements and tests the optimizer hints MERGE and NO_MERGE, table-level
hints to enable and disable, respectively, the derived_merge optimization
which is typically controlled by the derived_merge optimizer switch.
Sometimes hints need to be fixed before TABLE instances are available, but
after TABLE_LIST instances have been created (as in the cases of MERGE and
NO_MERGE). This commit introduces a new function called
fix_hints_for_derived_table to allow early hint fixing for derived tables,
using only a TABLE_LIST instance (so long as such hints are not index-level).
With strict SQL_MODE warnings generated during DDL statements
are threated as errors. This is done to prevent potential data
corruption. However, optimizer hints cannot affect the integrity
of data, so warnings during their parsing or application should not
be escalated to the level of errors.
This commit introduces `push_warning_safe()` method that guarantees
that a warning is not treated as an error, and generation of warnings
during hints processing now uses this method instead of traditional
`push_warning_printf()`
This commit implements optimizer hints allowing to affect the order
of joining tables:
- JOIN_FIXED_ORDER similar to existing STRAIGHT_JOIN hint;
- JOIN_ORDER to apply the specified table order;
- JOIN_PREFIX to hint what tables should be first in the join;
- JOIN_SUFFIX to hint what tables should be last in the join.
- remove get_args_printer() from hints printing
- add append_hint_arguments(THD *thd, opt_hints_enum hint, String *str)
- add more comments
- rename st_opt_hint_info::hint_name to hint_type
- add pptimizer trace support for hints
- add dbug_print_hints()
- make print_warn() not be a template
- introduce Printable_parser_rule interface, make grammar rules that
emit warnings implement it and print_warn invokes its function)
- remove Parser::Hint::append_args() as it is not used anywhere
(it used to be necessary call print_warn(... (Parser::Hint*)NULL);
- add a comment that opt_hints_global->check_unresolved() is never called
- improve comments
- rename everything with "resolved_children" to "fully_resolved_children"
- Opt_hints_table::adjust_key_hints() now returns value
- less "reach-back-to-parent" logic
- rename Hint "adjustment" and "resolution" (yes, both terms were used)
to "fixing". "Resolution" is already used for parse-tree objects
It places a limit N (a timeout value in milliseconds) on how long
a statement is permitted to execute before the server terminates it.
Syntax:
SELECT /*+ MAX_EXECUTION_TIME(milliseconds) */ ...
Only top-level SELECT statements support the hint.
Forbid adding optimizer hints to view definitions.
In the case when optimizer hints are added to the view definition
at a `CREATE (OR REPLACE) VIEW`/`ALTER VIEW` statement, a warning is
generated and the hints are ignored.
This commit also disables ps-protocol for test cases where
`Unresolved table/index name` warnings are generated. The reason
for this is such warnings are generated during both PREPARE
and EXECUTE stages. Since opt_hints.test has `--enable_prepare_warnings`,
running it with `--ps-protocol` causes duplication of warning messages
- Using Lex_ident_sys to scan identifiers, like the SQL parser does.
This fixes handling of double-quote-delimited and backtick-delimited identifiers,
as well as handling of non-ASCII identifiers.
Unescaping and converting from the client character set to the system
character set is now done using Lex_ident_cli_st and Lex_ident_sys,
like it's done in the SQL tokenizer/parser.
Adding helper methods to_ident_cli() and to_ident_sys()
in Optimizer_hint_parser::Token.
- Fixing the hint parser to report a syntax error when an empty identifiers:
SELECT /*+ BKA(``) */ * FROM t1;
- Moving a part of the code from opt_hints_parser.h to opt_hints_parser.cc
Moving these method definitions:
- Optimizer_hint_tokenizer::find_keyword()
- Optimizer_hint_tokenizer::get_token()
to avoid huge pieces of the code in the header file.
- A Lex_ident_cli_st cleanup
Fixing a few Lex_ident_cli_st methods to return Lex_ident_cli_st &
instead of void, to use them easier in the caller code.
- Fixing the hint parser to display the correct line number
Adding a new data type Lex_comment_st
(a combination of LEX_CSTRING and a line number)
Using it in sql_yacc.yy
- Getting rid of redundant dependencies on sql_hints_parser.h
Moving void LEX::resolve_optimizer_hints() from sql_lex.h to sql_lex.cc
Adding a class Optimizer_hint_parser_output, deriving from
Optimizer_hint_parser::Hint_list. Fixing the hint parser to
return a pointer to an allocated instance of Optimizer_hint_parser_output
rather than an instance of Optimizer_hint_parser::Hint_list.
This allows to use a forward declaration of Optimizer_hint_parser_output
in sql_lex.h and thus avoid dependencies on sql_hints_parser.h.
This commit introduces:
- the infrastructure for optimizer hints;
- hints for join buffering: BNL(), NO_BNL(), BKA(), NO_BKA();
- NO_ICP() hint for disabling index condition pushdown;
- MRR(), MO_MRR() hint for multi-range reads control;
- NO_RANGE_OPTIMIZATION() for disabling range optimization;
- QB_NAME() for assigning names for query blocks.
