Now that we do not pollute systemd's environment but write private
environment files running these as root is not longer required. So
let's drop `PermissionsStartOnly=true`.
Debian adds extra `ExecStartPre=` and `ExecStartPost=`, though.
Use special executable prefix for full privileges there. (See
systemd.service(5) for details.)
MDEV-32188 make TIMESTAMP use whole 32-bit unsigned range
This is done by changing my_time_t from long to unsigned long.
The effect of this is that on windows compling old clients may
get warnings of if they compare my_time_t with as signed variable.
Other things
- Removed my_time_t from include/*.pp files as it is different on windows
and linux.
- Changed do_abi_check.cmake to first print abi_check and then the
conflicting file (this makes it easier to find the cause of the error).
- ZLIB_LIBRARIES, not ZLIB_LIBRARY
- ZLIB_INCLUDE_DIRS, not ZLIB_INCLUDE_DIR
For building libmariadb, ZLIB_LIBRARY/ZLIB_INCLUDE_DIR are still defined
This workaround will be removed later.
Improve detection for DES support in OpenSSL, to allow compilation
against system OpenSSL without DES.
Note that MariaDB needs to be compiled against OpenSSL-like library
that itself has DES support which cmake detected. Positive detection
is indicated with CMake variable HAVE_des 1.
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@surgut.co.uk>
`FindZLIB` module uses variable `ZLIB_ROOT`[1] to look for libraries. By
setting the variable, `FindZLIB` is able to search the libraries that
installed in a non-system path (/workspace/mylib for example).
And when using `z` in `LINK_LIBRARIES()` CMake tries to lookup the
library in system path by default. It doesn't work if the library isn't
installed in the path, and use ${ZLIB_LIBRARY} which set by FindZLIB
solve the issue.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services.
[1]: https://cmake.org/cmake/help/latest/module/FindZLIB.html#hints
The libpmem dependency that had been added in
commit 3daef523af (MDEV-17084)
did not achieve any measurable performance improvement when
comparing the same PMEM device with and without "mount -o dax"
using the Linux ext4 file system.
Because Red Hat has deprecated libpmem, let us remove the code
altogether.
Note: This is a 10.6 version of
commit 3f9f5ca48e
which will retain PMEM support in MariaDB Server 10.11.
Because the Red Hat Enterprise Linux 8 core repository does not include
libpmem, let us implement the necessary subset ourselves.
pmem_persist(): Implement for 64-bit x86, ARM, POWER, RISC-V, Loongarch
in a way that should be compatible with the https://github.com/pmem/pmdk/
implementation of pmem_persist().
The CMake option WITH_INNODB_PMEM can be used for enabling or disabling
this interface at compile time. By default, it is enabled on all applicable
systems that are covered by our CI system.
Note: libpmem had not been previously enabled for Loongarch in our
Debian packaging. It was enabled for RISC-V, but we will not enable it
by default on RISC-V or Loongarch because we lack CI coverage.
The generated code for x86_64 was reviewed and tested on two
Intel implementations: one that only supports clflush, and
another that supports both clflushopt and clwb.
The generated machine code was also reviewed on https://godbolt.org
using various compiler versions. Godbolt helpfully includes an option
to compile to binary code and display the encoding, which was
useful on POWER.
Reviewed by: Vladislav Vaintroub
Some fixes related to commit f838b2d799 and
Rows_log_event::do_apply_event() and Update_rows_log_event::do_exec_row()
for system-versioned tables were provided by Nikita Malyavin.
This was required by test versioning.rpl,trx_id,row.
This was the orginal implementation that reverted with a bunch of
commits.
This reverts commit a13e521bc5.
Revert "cmake: append to the array correctly"
This reverts commit 51e3f1daf5.
Revert "build failure with cmake < 3.10"
This reverts commit 49cf702ee5.
Revert "MDEV-33301 memlock with systemd still not working"
This reverts commit 8a1904d782.
This commit removes the WITH_SSL=<custom_location_of_openssl> option,
leaving only -DWITH_SSL=bundled/system.
The rationale behind this removal is as follows:
1. The WITH_SSL=<custom_location_of_openssl> option is obscure
and not widely used.
2. There is no added value in this option compared to using
OPENSSL_ROOT_DIR. In fact, the availability of "helpful" MySQL options
might discourage users from exploring proper CMake options independently.
3. Users may incorrectly assume full MySQL compatibility even with this
option, including undocumented behaviors such as MySQL's preference for static libraries
with WITH_SSL=<custom_location_of_openssl>.
This change simplifies the configuration options and encourages users to
adopt more standardized and documented practices.
Apparently, invoking fcntl(fd, F_SETFL, O_DIRECT) will lead to
unexpected behaviour on Linux bcachefs and possibly other file systems,
depending on the operating system version. So, let us avoid doing that,
and instead just attempt to pass the O_DIRECT flag to open(). This should
make us compatible with NetBSD, IBM AIX, as well as Solaris and its
derivatives.
We will only implement innodb_log_file_buffering=OFF on systems where
we can determine the physical block size (typically 512 or 4096 bytes).
Currently, those operating systems are Linux and Microsoft Windows.
HAVE_FCNTL_DIRECT, os_file_set_nocache(): Remove.
OS_FILE_OVERWRITE, OS_FILE_CREATE_PATH: Remove (never used parameters).
os_file_log_buffered(), os_file_log_maybe_unbuffered(): Helper functions.
os_file_create_func(): When applicable, initially attempt to open files
in O_DIRECT mode. For type==OS_LOG_FILE && create_mode != OS_FILE_CREATE
we will first invoke stat(2) on the file name to find out if the size
is compatible with O_DIRECT. If create_mode == OS_FILE_CREATE, we will
invoke fstat(2) on the created log file afterwards, and may close and
reopen the file in O_DIRECT mode if applicable.
create_temp_file(): Support O_DIRECT. This is only used if O_TMPFILE is
available and innodb_disable_sort_file_cache=ON (non-default value).
Notably, that setting never worked on Microsoft Windows.
row_merge_file_create_mode(): Split from row_merge_file_create_low().
Create a temporary file in the specified mode.
Reviewed by: Vladislav Vaintroub
Apparently, invoking fcntl(fd, F_SETFL, O_DIRECT) will lead to
unexpected behaviour on Linux bcachefs and possibly other file systems,
depending on the operating system version. So, let us avoid doing that,
and instead just attempt to pass the O_DIRECT flag to open(). This should
make us compatible with NetBSD, IBM AIX, as well as Solaris and its
derivatives.
This fix does not change the fact that we had only implemented
innodb_log_file_buffering=OFF on systems where we can determine the
physical block size (typically 512 or 4096 bytes).
Currently, those operating systems are Linux and Microsoft Windows.
HAVE_FCNTL_DIRECT, os_file_set_nocache(): Remove.
OS_FILE_OVERWRITE, OS_FILE_CREATE_PATH: Remove (never used parameters).
os_file_log_buffered(), os_file_log_maybe_unbuffered(): Helper functions.
os_file_create_simple_func(): When applicable, initially attempt to
open files in O_DIRECT mode.
os_file_create_func(): When applicable, initially attempt to
open files in O_DIRECT mode.
For type==OS_LOG_FILE && create_mode != OS_FILE_CREATE
we will first invoke stat(2) on the file name to find out if the size
is compatible with O_DIRECT. If create_mode == OS_FILE_CREATE, we will
invoke fstat(2) on the created log file afterwards, and may close and
reopen the file in O_DIRECT mode if applicable.
create_temp_file(): Support O_DIRECT. This is only used if O_TMPFILE is
available and innodb_disable_sort_file_cache=ON (non-default value).
Notably, that setting never worked on Microsoft Windows.
row_merge_file_create_mode(): Split from row_merge_file_create_low().
Create a temporary file in the specified mode.
Reviewed by: Vladislav Vaintroub
- Use "new" math library WOLFSSL_SP_MATH_ALL, which is now promoted by
WolfSSL for faster performance. "fastmath" we used previously is going
to be deprecated, it was not really always fast.
- Optimize common RSA math operations with WOLFSSL_HAVE_SP_RSA
- Incorporate assembly optimizations, currently for Intel x64 only
This patch significantly reduces execution time for SSL tests like
main.ssl-big and main.ssl_connect, which now run 2 to 3 times faster.
Notably, when this patch is applied to 11.4, server startup in with
ephemeral certificates becomes approximately 10x faster due to optimized
wolfSSL_EVP_PKEY_keygen().
Additionally, refactored WolfSSL by removing old workarounds and
consolidating wolfssl and wolfcrypt into a single library wolfssl, just
like it was done in WolfSSL's own CMake.
CapabilityBoundingSet included CAP_IPC_LOCK in MDEV-9095, however
it requires that the executable has the capability marked in extended
attributes also.
The alternate to this is raising the RLIMIT_MEMLOCK for the service/
process to be able to complete the mlockall system call. This needs to
be adjusted to whatever the MariaDB server was going to allocate.
Rather than leave the non-obvious mapping of settings and tuning,
add the capability so its easier for the user.
We set the capability, if possible, but may never be used depending
on user settings. As such in the Debian postinst script, don't
complain if this fails.
The CAP_IPC_LOCK also facilitates the mmaping of huge memory pages.
(see man mmap), like mariadb uses with --large-pages.
X509_check_host() and X509_check_ip_asc() exist in all
supported SSL libraries
in OpenSSL >= 1.0.2 and in the bundled WolfSSL
And X509_free() handles NULL pointers all right.
With this patch, 4-component MSI version can be used, e.g by setting
TINY_VERSION variable in CMake, or by adding a string, e.g
MYSQL_VERSION_EXTRA=-2
which sets TINY_VERSION to 2, and also changes the package name.
The 4-component MSI versions do not support MSI major upgrades, only minor
ones, i.e do not reinstall components, just update existing ones based
on versioning rules.
To support these rules, add DefaultVersion for the files that won't
otherwise be versioned - headers, static and import libraries,
pdbs, text - xml, python and perl scripts Also silence WiX warning
that MSI won't store hashes for those files anymore.
When CMAKE_OSX_ARCHITECTURES isn't set we end up with
"Packaging as: mariadb-10.4.33-osx10.19-x86_64" on arm64 builders.
Instead of implying 64bit is x86, use CMAKE_SYSTEM_PROCESSOR to form
the filename.
The directio(3C) function on Solaris is supported on NFS and UFS
while the majority of users should be on ZFS, which is a copy-on-write
file system that implements transparent compression and therefore
cannot support unbuffered I/O.
Let us remove the call to directio() and simply treat
innodb_flush_method=O_DIRECT in the same way as the previous
default value innodb_flush_method=fsync on Solaris. Also, let us
remove some dead code around calls to os_file_set_nocache() on
platforms where fcntl(2) is not usable with O_DIRECT.
On IBM AIX, O_DIRECT is not documented for fcntl(2), only for open(2).
AIX compilation failed, because glibc's non-standard extension to
`struct tm` were used - additional members tm_gmtoff and tm_zone.
The patch fixes it by adding corresponding compile-time check.
Additionally, for the calculation of GMT offset on AIX, a portable
variant of timegm() was required.Implementation here is inspired by
SergeyD's answer on Stackoverflow :
https://stackoverflow.com/questions/16647819/timegm-cross-platform
Remove alarm() remnants
- Replace thread-unsafe use of alarm() inside my_lock.c with a
timed loop.
- Remove configure time checks
- Remove mysys my_alarm.c/my_alarm.h
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.
Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue
Disabled test:
- spider/bugfix.mdev_27239 because we started to get
+Error 1429 Unable to connect to foreign data source: localhost
-Error 1158 Got an error reading communication packets
- main.delayed
- Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
This part is disabled for now as it fails randomly with different
warnings/errors (no corruption).
Refactor CPackWixConfig.cmake so that it complains about unknown CPack
components.
This would prevent missing components,or discrepancy between MSI and ZIP
content, whenever new components are added or renamed.
Also, remove some old stuff, and use macros add_component_group() and
add_component() to reduce the verbosity.
KDF(key_str, salt [, {info | iterations} [, kdf_name [, width ]]])
kdf_name is "hkdf" or "pbkdf2_hmac" (default).
width (in bits) can be any number divisible by 8,
by default it's taken from @@block_encryption_mode
iterations must be positive, and is 1000 by default
OpenSSL 1.0 doesn't support HKDF, so it'll return NULL.
This OpenSSL version is still used in SLES 12 and CentOS 7
According to commit ea56841997
the stack normally grows downwards, except on HP PA-RISC where
it grows upwards. Because determining the stack direction is not
possible in a portable way, let us determine the default STACK_DIRECTION
in CMake based on the ISA.
On clang 16.0.6 running on and targeting AMD64, STACK_DIRECTION=1 is
being incorrectly detected, causing the failure of a number of tests.
`fmt::detail::make_arg` does not accept temporaries, so the code snippet
checking system libfmt needs to be adjusted.
Signed-off-by: Ruoyu Zhong <zhongruoyu@outlook.com>
MariaRocks is currently lagging behind the main branch of the RocksDB engine. This commit brings MariaRocks up to date with the latest release of RocksDB by backporting changes from Facebook’s MyRocks. These changes include API updates, bug fixes, and improvements for compatibility with RocksDB v8.1.1. Some system variables and metadata tables are modified to reflect the internal changes in RocksDB.
Additionally, this commit backports improved and more stable test cases from Facebook’s MyRocks, including tests for the write_unprepared isolation level of RocksDB. It also reverts workarounds for MDEV-29875 and MDEV-31057 and adds support for the latest compilation options.
The default value of the following system variables are changed:
* rocksdb_stats_level: 1 (kExceptHistogramOrTimers)
* rocksdb_wal_recovery_mode: 2 (kPointInTimeRecovery)
The following system variables are added:
* rocksdb_cancel_manual_compactions
* rocksdb_enable_iterate_bounds
* rocksdb_enable_pipelined_write
* rocksdb_enable_remove_orphaned_dropped_cfs
* rocksdb_manual_compaction_bottommost_level
* rocksdb_max_background_compactions
* rocksdb_max_background_flushes
* rocksdb_max_bottom_pri_background_compactions
* rocksdb_skip_locks_if_skip_unique_check
* rocksdb_track_and_verify_wals_in_manifest
* rocksdb_write_batch_flush_threshold
The following system variables are deprecated:
* rocksdb_hash_index_allow_collision
* rocksdb_new_table_reader_for_compaction_inputs
The following dynamic metadata table is added:
* INFORMATION_SCHEMA.ROCKSDB_LIVE_FILES_METADATA
The following status variables are added:
* rocksdb_manual_compactions_cancelled
* rocksdb_manual_compactions_pending
The following status variables are removed:
* rocksdb_block_cache_filter_bytes_evict
* rocksdb_block_cache_index_bytes_evict
* rocksdb_block_cachecompressed_hit
* rocksdb_block_cachecompressed_miss
* rocksdb_no_file_closes
* rocksdb_num_iterators
* rocksdb_number_deletes_filtered
* rocksdb_write_timedout
file /usr/lib64/libmariadb.so.3 from install of MariaDB-shared-10.11.3-1.fc38.x86_64 conflicts with file from package mariadb-connector-c-3.3.5-1.fc38.x86_64
These are mainly internal files so is a low impact change.
The few scripts/mysql*sql where renames to mariadb_* prefix
on the name.
mysql-test renamed to mariadb-test in the final packages
For both Deb and RPM, create mariadb-client-compat and
mariadb-server-compat containing the mysql links to the mariadb
named executables/scripts.
The mariadb-client-core mysqlcheck was moved to mariadb-client-compat.
The symlinks in MYSQL_ADD_EXECUTABLE is tagged as a
{Client,Server}Symlinks component and placed in
the symlinks packages.
Man pages are restructured be installed into compat package
if that matches the executable.
Columnstore has a workaround as it doesn't use the cmake/plugin.cmake.
Scripts likewise have compatibility symlinks are in
the {server,client}-compat packages.
Co-author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Closes#2390
* move them from ManPagesX component to X (works better for plugins),
but keep ManPagesDevelopment as C/C is using it
* move backup manpages to Backup
* move plugin manpages (s3, rocksdb) to plugins
During testing of RPM packages in MDEV-30203:
file /usr/share/man/man3 from install of
MariaDB-devel-11.0.1-1.el7_9.x86_64 conflicts with file from
package filesystem-3.2-25.el7.x86_64
MariaDB is the first libmariadb to include man3 man pages
so make the changes here like what is done for man1 and man8.
The MDEV-25004 test innodb_fts.versioning is omitted because ever since
commit 685d958e38 InnoDB would not allow
writes to a database where the redo log file ib_logfile0 is missing.
When building with -DWITH_WSREP=OFF, files required for MTR tests are
excluded and several tests fail. This is cause by a recent commit
7b44d0ba which attempted to resolve MDEV-23230.
Even when building without WSREP/Galera support some of the MTR include
files named *wsrep* are required by other tests.
Removing the following from the CMake install macros will avoid
excluding the MTR test .inc files:
`|include/((w.*)?wsrep.*|.*galera.*)\\.inc`
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Prevent wsrep files from being installed if WITH_WSREP=OFF.
Reviewed by Daniel Black
Additionally excluded #include wsrep files and galera* files
along with galera/wsrep tests.
mysql-test/include/have_wsrep.inc remainds as its used by
a few isolated tests.
Co-authored-by: Chris Ross <cross2@cisco.com>
with C/C.
The patch introduces mariadb_capi_rename.h which is included into
mysql.h. The hew header contains macro definitions for the names being
renamed. In versions 10.6+(i.e. where sql service exists) the renaming
condition in the mariadb_capi_rename.h should be added with
&& !defined(MYSQL_DYNAMIC_PLUGIN)
and look like
The patch also contains removal of mysql.h from the api check.
Disabling false_duper-6543 test for embedded.
ha_federated.so uses C API. C API functions are being renamed in the server,
but not renamed in embedded, since embedded server library should have proper
C API, as expected by programs using it.
Thus the same ha_federated.so cannot work both for server and embedded
server library.
As all federated tests are already disabled for embedded,
federated isn't supposed to work for embedded anyway, and thus the test
is being disabled.
