Previously a pair of days, I’ve been experimenting with Bazel as a library distribution mechanism for ccv.
I am reasonably conscious of hermetic opt up methods at this level. My foremost data comes from Buck relationship 8 years wait on. For the time being, it never occurred to me this kind of opt up draw may possibly possibly perhaps within the shatter be a library distribution mechanism. All the map throughout the equal 8 years, NPM has taken over the enviornment. Original language-dependent kit managers comparable to Hotfoot module, Cargo and Swift Equipment Supervisor popularized the thought that of utilizing the public repositories (GitHub) because the dependency references. Languages earlier than this period, mainly C / C++ are transferring to this route, slowly.
ccv has a easy autoconf primarily based mostly feature detection / configuration draw. You may possibly possibly count on the kit to work when
./configure && produce. Alternatively, it never made any excessive strive to be too natty. My initial trip with monorepos at firms strongly influenced the choice to maintain a easy opt up draw. I fully count on that excessive customers will dealer the library into their monorepo utilizing their own opt up methods.
This has been correct for the past a pair of years. But as I am finishing up nnc and extra and extra utilizing that for loads of closed-source private projects, putting ahead a closed-source monorepo setup for my private projects while upstreaming fixes is terribly an unsuitable trip. On the totally different hand, nnc from the originate meant to be a low-stage implementation. I am anticipated to maintain high-stage language bindings at some level. Given that I am doing extra utility-linked fashion with nnc in closed-source format now, it feels worship the correct time.
Even supposing there is no one-correct library distribution mechanism for C / C++, there are contenders. From the accurate-outmoded gleaming / rpm, to Conan, which has obtained some mind-section within the originate-source world recently.
The selection of Bazel is no longer accidental. I’ve been doing some Swift fashion with Bazel and the trip has been sure. Moreover, the selection of high-stage binding language for nnc, I figured, would be Swift.
ccv’s opt up route of, as noteworthy as I’d pretty no longer, is host-dependent. I take advantage of autoconf to detect draw-huge libraries comparable to libjpeg and libpng, to configure proper compiler suggestions. Even supposing ccv can even be weak with zero dependency, in that configuration, it would on occasion be slack.
Coming from the monorepo background, Bazel doesn’t maintain many utilities that are as readily accessible as in autoconf. You may possibly possibly also write automatic configurations in Starlark as repository principles, but there is not this kind of thing as a correct documentation on be taught the answer to jot down strong ones.
I stopped up letting whoever use ccv to resolve how they are going to enable obvious parts. For things worship CUDA, such configuration is no longer tenable. I stopped up copying over TensorFlow’s CUDA principles.
Appropriate outmoded C / C++ libraries are notoriously indifferent to libraries dependencies v.s. toolchains. Autoconf detects both toolchain configurations moreover to accessible libraries. All these host dependencies produce ugly-compilation a skill in itself.
Bazel is unbelievable for in-tree dependencies. For out-tree dependencies nonetheless, there is not this kind of thing as a established mechanism. The trendy technique is to jot down a repository principles to load linked dependencies.
Expend Bazel Dependencies
Consumption of the packaged Bazel dependencies then becomes as easy as adding
git_repository to the
WORKSPACE and name proper
After packaging ccv with Bazel, now Swift for nnc can luxuriate in the packaged dependency.
Semantic Versioning Challenges
Whereas the Bazel-supplied library distribution mechanism works successfully for my case, it’s simplistic. For one, there is de facto no correct technique to settle out semantic versioning. It is a long way understandable. Coming from a monorepo tradition, it’s tough for anyone to dive into dependency hells of library versioning. A a dinky of totally different myth came about to Hotfoot some time wait on as successfully.
It is a long way messy whenever you happen to would favor to pin a particular version of the library while your dependencies are no longer agreeing with you. Here goes to be messy regardless in C / C++ world, until you prelink these extremely fastidiously. Bazel’s philosophy from what I will gaze, appears largely on keeping the trunk working facet. It is a long way working to this level, but one has to surprise if this can scale if extra libraries adopted Bazel because the distribution mechanism.
The past a pair of months trip with Bazel has been delectable. Whereas I’d continue to utilize language particular instruments (pip, Hotfoot modules, Cargo, NPM) when doing fashion in that particular person language, Bazel is a capable want for me when doing ugly-language fashion. Concepts comparable to
http_archive fit successfully all the map throughout the larger originate-source ecosystem. And most surprisingly, it surely works for quite a bit of-repo setup whenever you happen to ever maintain to.