RFC 0040: Libstd facade

libs ()

Summary

Split the current libstd into component libraries, rebuild libstd as a facade in front of these component libraries.

Motivation

Rust as a language is ideal for usage in constrained contexts such as embedding in applications, running on bare metal hardware, and building kernels. The standard library, however, is not quite as portable as the language itself yet. The standard library should be as usable as it can be in as many contexts as possible, without compromising its usability in any context.

This RFC is meant to expand the usability of the standard library into these domains where it does not currently operate easily

Detailed design

In summary, the following libraries would make up part of the standard distribution. Each library listed after the colon are the dependent libraries.

libmini

Note: The name libmini warrants bikeshedding. Please consider it a placeholder for the name of this library.

This library is meant to be the core component of all rust programs in existence. This library has very few external dependencies, and is entirely self contained.

Current modules in std which would make up libmini would include the list below. This list was put together by actually stripping down libstd to these modules, so it is known that it is possible for libmini to compile with these modules.

This list may be a bit surprising, and it's makeup is discussed below. Note that this makeup is selected specifically to eliminate the need for the dreaded "one off extension trait". This pattern, while possible, is currently viewed as subpar due to reduced documentation benefit and sharding implementation across many locations.

Strings

In a post-DST world, the string type will actually be a library-defined type, Str (or similarly named). Strings will no longer be a language feature or a language-defined type. This implies that any methods on strings must be in the same crate that defined the Str type, or done through extension traits.

In the spirit of reducing extension traits, the Str type and module were left out of libmini. It's impossible for libmini to support all methods of Str, so it was entirely removed.

This decision does have ramifications on the implementation of libmini.

Formatting

While not often thought of as "ultra-core" functionality, this module may be necessary because printing information about types is a fundamental problem that normally requires no dependencies.

Inclusion of this module is the reason why I/O is included in the module as well (or at least a few traits), but the module can otherwise be included with little to no overhead required in terms of dependencies.

Neither print! nor format! macros to be a part of this library, but the write! macro would be present.

I/O

The primary reason for defining the io module in the libmini crate would be to implement the fmt module. The ramification of removing strings was previously discussed for IoError, but there are further modifications that would be required for the io module to exist in libmini:

Slices

The only method lost on mutable slices would currently be the sorting method. This can be circumvented by implementing a sorting algorithm that doesn't require allocating a temporary buffer. If intensive use of a sorting algorithm is required, Rust can provide a libsort crate with a variety of sorting algorithms apart from the default sorting algorithm.

FromStr

This trait and module are left out because strings are left out. All types in libmini can have their implemention of FromStr in the crate which implements strings

Floats

This current design excludes floats entirely from libmini (implementations of traits and such). This is another questionable decision, but the current implementation of floats heavily leans on functions defined in libm, so it is unacceptable for these functions to exist in libmini.

Either libstd or a libfloat crate will define floating point traits and such.

Failure

It is unacceptable for Option to reside outside of libmini, but it is also also unacceptable for unwrap to live outside of the Option type. Consequently, this means that it must be possible for libmini to fail.

While impossible for libmini to define failure, it should simply be able to declare failure. While currently not possible today, this extension to the language is possible through "weak lang items".

Implementation-wise, the failure lang item would have a predefined symbol at which it is defined, and libraries which declare but to not define failure are required to only exist in the rlib format. This implies that libmini can only be built as an rlib. Note that today's linkage rules do not allow for this (because building a dylib with rlib dependencies is not possible), but the rules could be tweaked to allow for this use case.

tl;dr; The implementation of libmini can use failure, but it does not define failure. All usage of libmini would require an implementation of failure somewhere.

liblibc

This library will exist to provide bindings to libc. This will be a highly platform-specific library, containing an entirely separate api depending on which platform it's being built for.

This crate will be used to provide bindings to the C language in all forms, and would itself essentially be a giant metadata blob. It conceptually represents the inclusion of all C header files.

Note that the funny name of the library is to allow extern crate libc; to be the form of declaration rather than extern crate c; which is consider to be too short for its own good.

Note that this crate can only exist in rlib or dylib form.

liballoc

Note: This name liballoc is questionable, please consider it a placeholder.

This library would define the allocator traits as well as bind to libc malloc/free (or jemalloc if we decide to include it again). This crate would depend on liblibc and libmini.

Pointers such as ~ and Rc would move into this crate using the default allocator. The current Gc pointers would move to libgc if possible, or otherwise librustrt for now (they're feature gated currently, not super pressing).

Primarily, this library assumes that an allocation failure should trigger a failure. This makes the library not suitable for use in a kernel, but it is suitable essentially everywhere else.

With today's libstd, this crate would likely mostly be made up by the global_heap module. Its purpose is to define the allocation lang items required by the compiler.

Note that this crate can only exist in rlib form.

libcollections

This crate would not depend on libstd, it would only depend on liballoc and libmini. These two foundational crates should provide all that is necessary to provide a robust set of containers (what you would expect today). Each container would likely have an allocator parameter, and the default would be the default allocator provided by liballoc.

When using the containers from libcollections, it is implicitly assumed that all allocation succeeds, and this will be reflected in the api of each collection.

The contents of this crate would be the entirety of libcollections as it is today, as well as the vec module from the standard library. This would also implement any relevant traits necessary for ~[T].

Note that this crate can only exist in rlib form.

libtext

This crate would define all functionality in rust related to strings. This would contain the definition of the Str type, as well as implementations of the relevant traits from libmini for the string type.

The crucial assumption of this crate is that allocation does not fail, and the rest of the string functionality could be built on top of this. Note that this crate will depend on libcollections for the Vec type as the underlying building block for string buffers and the string type.

This crate would be composed of the str, ascii, and unicode modules which live in libstd today, but would allow for the extension of other text-related functionality.

librustrt

This library would be the crate where the rt module is almost entirely implemented. It will assume that allocation succeeds, and it will assume a libc implementation to run on.

The current libstd modules which would be implemented as part of this crate would be:

Note that comm is not on this list. This crate will additionally define failure (as unwinding for each task). This crate can exist in both rlib and dylib form.

libsync

This library will largely remain what it is today, with the exception that the comm implementation would move into this crate. The purpose of doing so would be to consolidate all concurrency-related primitives in this crate, leaving none out.

This crate would depend on the runtime for task management (scheduling and descheduling).

The libstd facade

A new standard library would be created that would primarily be a facade which would expose the underlying crates as a stable API. This library would depend on all of the above libraries, and would predominately be a grouping of pub use statements.

This library would also be the library to contain the prelude which would include types from the previous crates. All remaining functionality of the standard library would be filled in as part of this crate.

Note that all rust programs will by default link to libstd, and hence will transitively link to all of the upstream crates mentioned above. Many more apis will be exposed through libstd directly, however, such as HashMap, Arc, etc.

The exact details of the makeup of this crate will change over time, but it can be considered as "the current libstd plus more", and this crate will be the source of the "batteries included" aspect of the rust standard library. The API (reexported paths) of the standard library would not change over time. Once a path is reexported and a release is made, all the path will be forced to remain constant over time.

One of the primary reasons for this facade is to provide freedom to restructure the underlying crates. Once a facade is established, it is the only stable API. The actual structure and makeup of all the above crates will be fluid until an acceptable design is settled on. Note that this fluidity does not apply to libstd, only to the structure of the underlying crates.

Updates to rustdoc

With today's incarnation of rustdoc, the documentation for this libstd facade would not be as high quality as it is today. The facade would just provide hyperlinks back to the original crates, which would have reduced quantities of documentation in terms of navigation, implemented traits, etc. Additionally, these reexports are meant to be implementation details, not facets of the api. For this reason, rustdoc would have to change in how it renders documentation for libstd.

First, rustdoc would consider a cross-crate reexport as inlining of the documentation (similar to how it inlines reexports of private types). This would allow all documentation in libstd to remain in the same location (even the same urls!). This would likely require extensive changes to rustdoc for when entire module trees are reexported.

Secondly, rustdoc will have to be modified to collect implementors of reexported traits all in one location. When libstd reexports trait X, rustdoc will have to search libstd and all its dependencies for implementors of X, listing them out explicitly.

These changes to rustdoc should place it in a much more presentable space, but it is an open question to what degree these modifications will suffice and how much further rustdoc will have to change.

Remaining crates

There are many more crates in the standard distribution of rust, all of which currently depend on libstd. These crates would continue to depend on libstd as most rust libraries would.

A new effort would likely arise to reduce dependence on the standard library by cutting down to the core dependencies (if necessary). For example, the libnative crate currently depend on libstd, but it in theory doesn't need to depend on much other than librustrt and liblibc. By cutting out dependencies, new use cases will likely arise for these crates.

Crates outside of the standard distribution of rust will like to link to the above crates as well (and specifically not libstd). For example, crates which only depend on libmini are likely candidates for being used in kernels, whereas crates only depending on liballoc are good candidates for being embedded into other languages. Having a clear delineation for the usability of a crate in various environments seems beneficial.

Alternatives

Unresolved questions