~smspillaz/folly/folly-git-master : changes

~smspillaz/folly/folly-git-master » Changes from revision 9855

From Revision 8014 to 7995

Rev	Summary	Authors	Date
8014	fbcode_builder: getdeps: OpenNSA update source URL fbcode_builder: getdeps: OpenNSA update source URL Summary: This is an interim solution that unblocks failing FBOSS OSS builds. Broadcom's OpenNSA is available on github and contains large precompiled libraries. For large files (> 100Mb), github uses git-lfs. git clones to OpenNSA started failing yesterday with: Error downloading object: lib/x86-64/libopennsa.a (7f21d94): Smudge error: Error downloading lib/x86-64/libopennsa.a (7f21d941ac32ccae38adb05b3386be739bdccd502a878364d386e7a5cb172f35): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access. Github documents (refer: https://help.github.com/en/github/managing-large-files/about-storage-and-bandwidth-usage) “When you download a file tracked with Git LFS, the total file size is counted against the repository owner’s bandwidth limit”. It appears we are hitting that limit. Thus, github is not a good choice here. Even if we create a 'github release tarball', the tarball continues to carry lfs files and thus downloading those counts towards the quota (which is limited). Broadcom paid some amount to bump up the quota, but it is matter of time before we hit it esp given that these builds run on-diff. A better solution is to download a tarball (that contains real files, not lfs), and then allow Facebook lfs to cache it. While github does not support that, it seems bitbucket does. Thus, we forked OpenNSA from github into a bitbucket repo and this patch modifies the manifest to download a tarball from bitbucket that will be cached. Broadcom is working on a cleaner solution but this will keep our FBOSS OSS build clean in the interim. Reviewed By: wez Differential Revision: D19437386 fbshipit-source-id: b1213a186a0af57b9fd71b30e8899e80affd4cc0	Shrikrishna Khare	4 years ago
8013	hazptr: Make using a thread pool executor for default domain... hazptr: Make using a thread pool executor for default domain asynchronous reclamation the default Summary: Add a call in folly init to folly::start_hazptr_thread_pool_executor() to make the default hazptr domain use a CPUThreadPoolExecutor by default. Reviewed By: davidtgoldblatt Differential Revision: D19107125 fbshipit-source-id: 1cebe0864de14b54349f723061c4c2b5905b105b	Maged Michael	4 years ago
8012	Add overload of blockingWait() that takes DrivableExecutor Add overload of blockingWait() that takes DrivableExecutor Summary: This allows the caller to, for example, pass their own ManualExecutor or EventBase and have the task scheduled onto that executor and have the current thread drive that executor until it completes. Reviewed By: yfeldblum Differential Revision: D19205803 fbshipit-source-id: 6042f68decf8d7d9bb1d37977e5804b0504daa41	Lewis Baker	4 years ago
8011	Add socket to notifyDataAvailable Add socket to notifyDataAvailable Summary: add the socket to the callback parameter Reviewed By: yangchi Differential Revision: D19192549 fbshipit-source-id: 068bdb560273ab3203a1b7eaefa5b01c4e4e9968	Subodh Iyengar	4 years ago
8010	Add support for registered fds Add support for registered fds Summary: Add support for registered fds Reviewed By: kevin-vigor Differential Revision: D19313387 fbshipit-source-id: 19527958b3986992cdafbad2c4c4902c887a9e9f	Dan Melnic	4 years ago
8009	fbcode_builder: getdeps: fboss remove OpenNSL manifest fbcode_builder: getdeps: fboss remove OpenNSL manifest Summary: Broadcom has formally replaced OpenNSL with newer OpenNSA: https://github.com/Broadcom-Network-Switching-Software/OpenNSA/blob/master/README.md With D16401189, FBOSS no longer depends on OpenNSL, but uses the new OpenNSA instead. Remove this manifest. Reviewed By: wez Differential Revision: D19413708 fbshipit-source-id: f3662f1101d59a5e0e59cb7238797d652564ff90	Shrikrishna Khare	4 years ago
8008	Use getrandom instead of /dev/urandom when available Use getrandom instead of /dev/urandom when available Summary: [Folly] Use `getrandom` instead of `/dev/urandom` when available. The C library function `getrandom` is available on linux >= 3.17 and glibc >= 2.25. Reviewed By: ot, simpkins, luciang Differential Revision: D19211519 fbshipit-source-id: 07e7e856fc81d1f04efca4a6cfa0b76ada08f443	Yedidya Feldblum	4 years ago
8007	make msan happy with exception_wrapper make msan happy with exception_wrapper Summary: When reading a `std::exception_ptr` out of a moved-from `folly::exception_wrapper`, we end up reading a `std::exception_ptr` field of a deleted `ExceptionPtr` object. This was causing MSAN much consternation. The fix is to test for emptiness in `to_exception_ptr` and return a default-constructed `std::exception_ptr`. This commit also adds extra test cases, since `to_exception_ptr` does something slightly different depending on whether the `exception_wrapper` object is `const` or not. Reviewed By: yfeldblum Differential Revision: D19403528 fbshipit-source-id: fde74a5c4b2ddb2025703432c1ac5a19d26b9426	Eric Niebler	4 years ago
8006	prevent asan failures in SingletonThreadLocal prevent asan failures in SingletonThreadLocal Summary: I have been encountering ``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==937752==ERROR: AddressSanitizer: SEGV on unknown address 0x24a12000e4f18 (pc 0x7f3902a6e696 bp 0x7f37bc5e4380 sp 0x7f37bc5e42a0 T107) ==937752==The signal is caused by a READ memory access. SCARINESS: 20 (wild-addr-read) #0 0x7f3902a6e695 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::Wrapper, void, void>::get() const folly/ThreadLocal.h:160 #1 0x7f3902a6d600 in get folly/ThreadLocal.h:69 #2 0x7f3902a6d600 in folly::ThreadLocal<folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::Wrapper, void, void>::operator() const folly/ThreadLocal.h:78 #3 0x7f3902a6ca1b in folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::getWrapper() folly/SingletonThreadLocal.h:149 #4 0x7f3902a6cbe5 in folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::LocalLifetime::~LocalLifetime() folly/SingletonThreadLocal.h:121 #5 0x7f38f04d9fd5 in (anonymous namespace)::run(void) /data/users/kirkshoop/gcc/trunk/libstdc++-v3/libsupc++/atexit_thread.cc:75:16 #6 0x7f38ef9ee551 in __nptl_deallocate_tsd.part.8 /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/nptl/pthread_create.c:301:8 #7 0x7f38ef9ef7c8 in __nptl_deallocate_tsd /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/sysdeps/nptl/futex-internal.h:200:3 #8 0x7f38ef9ef7c8 in start_thread /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/nptl/pthread_create.c:475:3 ``` I tried using a unique Tag for the SingletonThreadLocal for debugging purposes and this seemed to fix the issue. Reviewed By: magedm Differential Revision: D19356179 fbshipit-source-id: a18c6b02f0a40e0509753075614eeff71d859bf7	Kirk Shoop	4 years ago
8005	OpenNSA: Don't use opennsl, switch to bcm-only API OpenNSA: Don't use opennsl, switch to bcm-only API Summary: With the release of OpenNSA (https://github.com/Broadcom-Network-Switching-Software/OpenNSA), we can stop using opennsal altogether! This diff was created using following steps: cd fboss/agent find . ! -path "\/oss" -type f \( ! -iname "TARGETS" \) -exec sed -i -e 's/opennsl_spl_//g' {} \; find . ! -path "\/oss" -type f \( ! -iname "TARGETS" \) -exec sed -i -e 's/opennsl/bcm/g' {} \; find . ! -path "\/oss" -type f \( ! -iname "TARGETS" \) -exec sed -i -e 's/OPENNSL/BCM/g' {} \; find . ! -path "\/oss" -type f \( ! -iname "TARGETS" \) -exec sed -i -e 's/OpenNSL/Bcm/g' {} \; # for symbols like snmpBcmTransmittedPkts2048to4095Octets Furthermore: - In all TARGETS under fboss/agent replace wrapped_opennsl_symbols with wrapped_bcm_symbols. - hg mv agent/facebook/test/OpenNSLInterface.h agent/facebook/test/BcmInterface.h - BcmEgress.cpp remove operator== overload for opennsl API altogether, and remove operator== BCM API version from inside unnamed namespace, and put it in fboss namespace. - BcmCinter.h and BcmCinter.cpp now have some duplicate definitions as opennsl_ got renamed to bcm_. Delete one of the definitions "as appropriate". - Similarly, remove duplicate definitions in FakeSdk.h and FakeSdk.cpp - Similarly, remove duplicate definitions in SdkTracer.cpp (remove definitions calling CALL_WRAPPERS_RV_NO_CINTER). And lastly, remove CALL_WRAPPERS_RV_NO_CINTER itself. - arc lint - Remove OpennslCompatTests.cpp - Fix ./agent/hw/bcm/tests/facebook/BcmEgressTest.cpp to have #include "fboss/agent/facebook/test/MockOpenNSL.h" instead of MockBcm.h TODO: - This patch retains the directory structure (facebook/ subdirectory), which will be fixed in subsequent patches. - Remove opennsl references from TARGETS file, additional cleanup etc. - Remove all oss directories... Differential Revision: D16401189 fbshipit-source-id: 607b6c0f97ffcacf3707f6b7a4c9454cc0b24476	Shrikrishna Khare	4 years ago
8004	fbcode_builder: getdeps: fboss: rename OpenBCM to OpenNSA fbcode_builder: getdeps: fboss: rename OpenBCM to OpenNSA Summary: Broadcom provides this library and they decided to rename it from OpenBCM to OpenNSA. Thus, rename corresponding fbcode_builder code. Reviewed By: wez Differential Revision: D19396687 fbshipit-source-id: 8233dbf4de9342b5a0e54ae275d6c73d43abe6d0	Shrikrishna Khare	4 years ago
8003	Cut unused AsyncSocketException::SASL_HANDSHAKE_TIMEOUT Cut unused AsyncSocketException::SASL_HANDSHAKE_TIMEOUT Summary: [Folly] Cut unused `AsyncSocketException::SASL_HANDSHAKE_TIMEOUT`. Reviewed By: vitaut Differential Revision: D19343864 fbshipit-source-id: ed4374bc2b376ff71e592323ed8c37290700e870	Yedidya Feldblum	4 years ago
8002	Add recvmmsg Add recvmmsg Summary: Add recvmmsg Differential Revision: D18927890 fbshipit-source-id: de500dd1c266f21fbfc39c2972df787da50c7a80	Konstantin Tsoy	4 years ago
8001	fix implicit-integer-truncation UBSAN error in AtomicUtil-in... fix implicit-integer-truncation UBSAN error in AtomicUtil-inl.h Summary: UndefinedBehaviorSanitizer: implicit-integer-truncation buck-out/dev/gen/folly/synchronization/atomic_util#header-mode-symlink-tree-with-header-map,headers/folly/synchronization/AtomicUtil-inl.h:115:28 ``` Here both `mask` and the first argument in `std::atomic<..>::fetch_and` have type `unsigned char`. Before bitwise not operation `mask` variable undergoes [integral promotion](https://en.cppreference.com/w/cpp/language/implicit_conversion#Integral_promotion) resulting in truncation error. Since here this is intentional add a `static_cast` to be explicit about it. Reviewed By: yfeldblum Differential Revision: D19320340 fbshipit-source-id: 41c987c19b77fa21ee9aa83cf06dfc7cddba91c8	Igor Sugak	4 years ago
8000	Simplify boolean expressions (#1301) Simplify boolean expressions (#1301) Summary: Found with readability-simplify-boolean-expr Signed-off-by: Rosen Penev <rosenp@gmail.com> Pull Request resolved: https://github.com/facebook/folly/pull/1301 Reviewed By: yfeldblum Differential Revision: D19302630 Pulled By: Orvid fbshipit-source-id: 9833e4ab3fe93592fa8f868f6ed2d56a263949fe	Rosen Penev	4 years ago
7999	getdeps: enable the INSTALL_PYTHON_LIB option for Eden build... getdeps: enable the INSTALL_PYTHON_LIB option for Eden builds on Windows Summary: Enable `INSTALL_PYTHON_LIB=ON` for internal builds of Eden on Windows. Reviewed By: pkaush Differential Revision: D19060630 fbshipit-source-id: 40d4f8784958fbeaad4f0d332c55fa0cb810cb5e	Adam Simpkins	4 years ago
7998	add FOLLY_ATTR_NO_UNIQUE_ADDRESS add FOLLY_ATTR_NO_UNIQUE_ADDRESS Summary: Add `FOLLY_NO_UNIQUE_ADDRESS ` to expose C++20's `[[no_unique_address]]` where available. (Note: this ignores all push blocking failures!) Reviewed By: pixelb Differential Revision: D19375089 fbshipit-source-id: 9a83ae791002e02df5b93b61f20209e5f467a959	Phil Willoughby	4 years ago
7997	disable VDSO getcpu optimization in MSAN build mode disable VDSO getcpu optimization in MSAN build mode Reviewed By: yfeldblum Differential Revision: D19359860 fbshipit-source-id: b084d8cd603238e76d61de1eed3f233cf5b7fb27	Igor Sugak	4 years ago
7996	disable VDSO clock_gettime optimization in MSAN build mode disable VDSO clock_gettime optimization in MSAN build mode Reviewed By: yfeldblum Differential Revision: D19359673 fbshipit-source-id: 610ae77e205038267e0e331a04486eef0329c165	Igor Sugak	4 years ago
7995	IOBuf: Add goodSize() Allocation-Sizing Method IOBuf: Add goodSize() Allocation-Sizing Method Summary: For long-lived IOBufs, it can be useful to know whether their memory footprint can be usefully shrunk. This diff exposes a method that lets users see the smallest capacity an IOBuf could have while wrapping a particularly sized object. Without this, they have to do hacky math. Reviewed By: yfeldblum Differential Revision: D18440359 fbshipit-source-id: 1324700b58f0b2762cb3347b907ca5e04ebecb99	Felix Handte	4 years ago