~brianaker/memcached/1.4 : changes

~brianaker/memcached/1.4 » Changes from revision 1414

From Revision 1414 to 1395

Rev	Summary	Authors	Date
1414	fix sasl tests fix sasl tests apparently I broke them with the unix domain socket update, since you have to add an environment variable for them to even run. need to make it work with the unix socket at some point though.	dormando	5 years ago
1413	extstore tests: further tuning of tests extstore tests: further tuning of tests enforce actually waiting for extstore to flush items before moving on, remove some pacing that was making things worse. The compactor is rescuing the canary items occasionally. Detunes the compactor on start, then ramps it up before the compactor specific tests. This seems to fix the flakiness, at least enough that it's been passing in a loop on fast and slow systems for me.	dormando	5 years ago
1412	extstore tests: adjust pacing and sizing extstore tests: adjust pacing and sizing on 32bit systems the items take a bit less space in a few dimensions, so we need to fill harder. also, these tend to be slower systems, so pace out the inserts with occasional 1s sleeps, rather than all at once at the end.	dormando	5 years ago
1411	gate arm crc32 behind --enable-arm-crc32 gate arm crc32 behind --enable-arm-crc32 users also need to add CFLAGS="-march=armv8-a+crc" if they have actual aarch64 platforms.	dormando	5 years ago
1410	extstore test: try harder to find missing values extstore test: try harder to find missing values	dormando	5 years ago
1409	alignment and 32bit fixes for extstore alignment and 32bit fixes for extstore memory alignment when reading header data back. left "32" in a few places that should've at least been a define, is now properly an offsetof. used for skipping crc32 for dynamic parts of the item headers.	dormando	5 years ago
1408	crc32c for aarch64 support crc32c for aarch64 support Patch authored by @aparida1 on github portability/compilation/warnings modifications by dormando	dormando	5 years ago
1407	fix deadlock during hash table expansion fix deadlock during hash table expansion The slab rebalancer was paused after the LRU maintainer thread during hash table expansion. This was mostly fine, except in one case. If an item is in the COLD LRU when it gets hit, it gets assigned an ACTIVE bitflag and queued for an asynchronous bump. If the item already has an ACTIVE bit, it does not queue a second time. Items in other LRU's are bumped when they hit the tail, but COLD items have to bump relatively soon to avoid accidentally evicting recently active items while under memory pressure. Items queued for async bumps have an extra refcount. The slab page mover cannot move a page until all memory within the page is cleared and freed. If it runs into an item which is stuck, it will try to delete it, then wait for all of its references to be removed. However, if an item is in the bump queue when the LRU thread is stopped, and the slab page mover is moving the page a queued item happens to be in, the refcount will never drop to zero as the bump will never happen. The page mover has no mechanism for "giving up" partway through, so it loops forever. For most use cases, this degrades the instance but doesn't actually break much: - The COLD LRU's will eventually drain to zero. - Allocations will all become "direct reclaims", which force items into COLD and then evicts them. - This will degrade the LRU algorithm and some performance, but if the instance usage isn't very high most users may never notice. - Normally, page moving is rare enough and hash table expansion is rare enough that it's an unlikely race. However, with extstore, page moves tend to happen a lot more frequently. Also, items are only ever written to extstore from the COLD LRU's, so if they empty the process will stop. This makes the race more likely to happen and the effects are more obvious. The fix is to simply pause the page mover before the LRU thread. The LRU thread does call the page mover, but it is via trylock() since it normally expects to progress even if the page mover is busy. The pause of the page mover won't succeed until it fully completes the current page, so it should then be clear to pause the LRU thread. Further improvements could be made by allowing the page mover to bail after N unsuccessful loops. I am also considering removing the async LRU bump process, and letting the LRU crawler handle that instead. Much thanks to Shashi and Sridhar from Netflix for all of their help in tracking down this bug!	dormando	5 years ago
1406	Add Dockerfile definitions Add Dockerfile definitions Add alpine (musl-based) and ubuntu (glibc-based) descriptions for test images. To run the testapp in either of them, run: ``` docker-compose run alpine docker-compose run ubuntu ```	Stanisław Pitucha	5 years ago
1405	Fix lru-crawler behaviour Fix lru-crawler behaviour Allow worker threads to poll	Stanisław Pitucha	6 years ago
1404	Fail loudly if seccomp setup fails Fail loudly if seccomp setup fails	Stanisław Pitucha	6 years ago
1403	extstore: fix ref leak when using binprot GATK extstore: fix ref leak when using binprot GATK GATK returns a key but not the value. c->io_wraplist is only appended if the value is to be returned, but c->item is skipped if it is an ITEM_HDR at all. This now checks for the ITEM_HDR bit being set but also !value which then reclaims the reference normally. I knew doubling up the cleanup code made it a lot more complex, and hope to flatten that to a single path. Also the TOUCH/GAT/GATK binprot code has no real test coverage, nor mc-crusher entries. Should be worth fixing.	dormando	6 years ago
1402	Bugfix: missing initialization code for extstore maint threa... Bugfix: missing initialization code for extstore maint thread objects The extstore maintenance thread takes permanent ownership of its mutex. Grabbing an uninitialized mutex can result in undefined behavior. In this case the memory is zeroed so probably no harm.	Ori Shalev	6 years ago
1401	Drop supplementary groups in addition to setgid Drop supplementary groups in addition to setgid When launching as root, we drop permissions to run as a specific user, however the way this is done fails to drop supplementary groups from the process. On many systems, the root user is a member of many groups used to guard important system files. If we neglect to drop these groups it's still possible for a compromised memcached instance to access critical system files as though it were running with root level permissions. On any given system `grep root /etc/group` will reveal all the groups root is a member of, and memcached will have access to any file these groups are authorized for, in spite of our attempts to drop this access. It's possible to test if a given memcached instance is affected by this and running with elevated permissions by checking the respective line in procfs: `grep Groups /proc/$pid/status` If any groups are listed, memcached would have access to everything the listed groups have access to. After this patch no groups will be listed and memcached will be locked down properly.	Anthony Ryan	6 years ago
1400	Use HAVE_SASL_CB_GETCONFPATH Use HAVE_SASL_CB_GETCONFPATH With SASL_CB_GETCONFPATH detection added, HAVE_SASL_CB_GETCONFPATH can be used. For https://github.com/memcached/memcached/issues/365 On RHEL(5\|6\|7) the name of the defined constant in sasl.h is SASL_CB_GETCONFPATH and not SASL_CB_GETCONF. This is used when ./configure is run with "--enable-sasl".	Peter (Stig) Edwards	6 years ago
1399	Fix SASL_CB_GETCONF(PATH) detection Fix SASL_CB_GETCONF(PATH) detection For https://github.com/memcached/memcached/issues/365 On RHEL(5\|6\|7) the name of the defined constant in sasl.h is SASL_CB_GETCONFPATH and not SASL_CB_GETCONF. This adds support for HAVE_SASL_CB_GETCONFPATH to be set to nonzero when the SASL implementation supports SASL_CB_GETCONFPATH. This will be used in sasl_defs.c when ./configure is run with "--enable-sasl".	Peter (Stig) Edwards	6 years ago
1398	memcached-tool: Add 'limit' parameter to 'dump'. memcached-tool: Add 'limit' parameter to 'dump'.	Calin Iorgulescu	6 years ago
1397	Rewrite memchached-tool 'dump' method to use new lru_crawler Rewrite memchached-tool 'dump' method to use new lru_crawler interface.	Calin Iorgulescu	6 years ago
1396	Fixes decrement-before-check problem (issue #362). Fixes decrement-before-check problem (issue #362). Adds test for issue #362.	Calin Iorgulescu	6 years ago
1395	document in manpage that port 0 is off. document in manpage that port 0 is off. issue #354	dormando	6 years ago

Older »