doc: Document lockfile importer based Rust packaging workflow.

* doc/guix.texi (Build Systems) [cargo-build-system]: Add cross-reference for
the term "Cargo workspaces".
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
* doc/guix-cookbook.texi (Packaging)[Packaging Workflow]: New section.
* gnu/packages/rust-crates.scm,
* gnu/packages/rust-sources.scm: Stop mentioning guix-rust-registry for now, we
may remove the repository if future merges are managed well.

Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc

doc/contributing.texi

@@ -1635,34 +1635,78 @@ dashes and prepend the prefix @code{java-}.  So the class
 @subsection Rust Crates
 
 @cindex rust
-Rust programs standing for themselves are named as any other package, using the
+Rust applications (binary crates) and libraries (library crates) are packaged
-lowercase upstream name.
+separately.  We put our main efforts into applications and only package
+libraries as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies.
 
-To prevent namespace collisions we prefix all other Rust packages with the
+Rust applications are treated like any other package and named using the
-@code{rust-} prefix.  The name should be changed to lowercase as appropriate and
+lowercase upstream name.  When using the Cargo build system (@pxref{Build
-dashes should remain in place.
+Systems, @code{cargo-build-system}}), Rust applications should have the
+@code{#:install-source?} parameter set to @code{#f}, as this parameter only
+makes sense for libraries.  When the package source is a Cargo workspace,
+@code{#:cargo-install-paths} must be set to enable relevant support.
 
-In the rust ecosystem it is common for multiple incompatible versions of a
+Rust libraries are hidden from the user interface and managed in two modules:
-package to be used at any given time, so all package definitions should have a
-versioned suffix.  The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course).  This follows the ``caret'' version scheme
-intended by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
 
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
+@table @code
-other packages the Cargo build system (@pxref{Build Systems,
+@item (gnu packages rust-crates)
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
+Source definitions imported from Rust packages' @file{Cargo.lock} via the
-@code{cargo-development-inputs} keywords as build system arguments.  It would be
+@code{crate} importer (@pxref{Invoking guix import}).
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}.  Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}.  If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
 
-Care should be taken to ensure the correct version of dependencies are used; to
+Imported definitions must be checked and have vendored dependencies unbundled
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
+before being contributed to Guix.
-when possible.  Of course this is not always possible, as the package may be
+
-developed for a different Operating System, depend on features from the Nightly
+@item (gnu packages rust-sources)
-Rust compiler, or the test suite may have atrophied since it was released.
+More complex definitions that need to be full packages.  This includes Rust
+libraries requiring external inputs to unbundle and Cargo workspaces.
+
+These libraries should have the @code{#:skip-build?} parameter set to @code{#t}.
+For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, they follow the usual naming convention for Guix
+packages (@pxref{Package Naming}), with a @code{rust-} prefix.
+
+In the Rust community it is common for multiple incompatible versions of a
+package to be used at any given time, so all libraries should have a versioned
+suffix.  The versioned suffix is the left-most non-zero digit (and any leading
+zeros, of course).  This follows the ``caret'' version scheme intended by Cargo.
+Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we often package development snapshots of Rust libraries and can't
+simply identify them by version.  In this case, the complete version information
+can be turned into a version string via @code{git-version}, for example
+@code{rust-pipewire-0.8.0.fd3d8f7} and @code{rust-pubgrub-0.3.0.b70cf70}.
+@end table
+
+To avoid merge conflicts with changes from multiple branches, these two modules
+are maintained by the Rust team (@pxref{Teams}).
+
+Rust libraries are not referenced directly.  @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to lookup input lists generated by the
+lockfile importer.
+
+@cindex cargo inputs
+@findex define-cargo-inputs
+@findex lookup-cargo-inputs
+@deffn {Procedure} cargo-inputs name [#:module '(gnu packages rust-crates)]
+Lookup Cargo inputs for @var{name} in @var{module}, return an empty list if
+unavailable.
+
+@var{name} must be consistent with the one used in lockfile importer invocation,
+usually matching the variable name of the package:
+
+@example
+$ guix import -i @file{gnu/packages/rust-crates.scm} crate -f @file{/path/to/Cargo.lock} @var{name}
+@end example
+
+@var{module} must export a public interface @code{lookup-cargo-inputs}, a
+template is available in the @file{etc/teams/rust} directory of Guix source
+tree.
+@end deffn
+
+@xref{Packaging Rust Crates,,, guix-cookbook, GNU Guix Cookbook}, for
+packaging workflow.
 
 
 @node Elm Packages

doc/guix-cookbook.texi

@@ -106,6 +106,7 @@ Scheme tutorials
 Packaging
 
 * Packaging Tutorial::          A tutorial on how to add packages to Guix.
+* Packaging Workflows::         Real life examples on working with specific build systems.
 
 Packaging Tutorial
 
@@ -130,6 +131,16 @@ Programmable and automated package definition
 * Automatic update::
 * Inheritance::
 
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Rust Packaging::
+* Cargo Workspaces and Development Snapshots::
+* Using Rust Libraries in Other Build Systems::
+
 System Configuration
 
 * Auto-Login to a Specific TTY::  Automatically Login a User to a Specific TTY
@@ -519,6 +530,7 @@ them.
 
 @menu
 * Packaging Tutorial::          A tutorial on how to add packages to Guix.
+* Packaging Workflows::         Real life examples on working with specific build systems.
 @end menu
 
 @node Packaging Tutorial
@@ -1599,6 +1611,371 @@ The @uref{https://guix.gnu.org/manual/en/html_node/Defining-Packages.html, packa
 @uref{https://guix.gnu.org/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
 @end itemize
 
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections provide real-life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Rust Packaging::
+* Cargo Workspaces and Development Snapshots::
+* Using Rust Libraries in Other Build Systems::
+@end menu
+
+@node Common Workflow for Rust Packaging
+@subsubsection Common Workflow for Rust Packaging
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository.  All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, we can generate a template
+via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU Guix
+Reference Manual}):
+
+@example
+$ guix import crate cargo-audit
+@end example
+
+After manual editing, we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+  (package
+    (name "cargo-audit")
+    (version "0.21.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (crate-uri "cargo-audit" version))
+       (file-name (string-append name "-" version ".tar.gz"))
+       (sha256
+        (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+    (build-system cargo-build-system)
+    (arguments (list #:install-source? #f))
+    (inputs (cargo-inputs 'cargo-audit))
+    (home-page "https://rustsec.org/")
+    (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+    (description
+     "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+    (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The identifier used to invoke @code{cargo-inputs}, in this case
+@code{'cargo-audit}, must be unique, usually matching the variable name of the
+package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then run the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions.
+Applying it to all Rust applications helps reduce a great number of Rust
+libraries we need to check later.  Although sometimes libraries may fail to
+follow the @uref{https://semver.org/, semantic versioning} scheme, it's still
+acceptable.
+
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses, for all the dependencies.  We must have an acceptable output of
+@command{cargo audit} and ensure all dependencies are licensed with our
+supported licenses (@pxref{Defining Packages,,, guix, GNU Guix Reference
+Manual}).
+
+@item
+Import dependencies from the generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+      crate --lockfile=/path/to/Cargo.lock cargo-audit
+
+# Or use short options, in this case the shell processes file names
+# before passing them to Guix, allowing tilde expansion, for example.
+$ guix import -i gnu/packages/rust-crates.scm \
+      crate -f /path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} here must be consistent with the identifier used for
+@code{cargo-inputs} invocation in the package definition.
+
+At this stage, the package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle the vendored dependencies.  The lockfile importer
+inserts @code{TODO:} comments for libraries with high probability of
+bundled dependencies.  @code{cargo-build-system} also performs
+additional check for binary files in its
+@code{check-for-pregenerated-files} phase, which usually indicates
+bundling:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_aclui.a
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_activeds.a
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_asycfilt.a
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_amsi.a
+@dots{}
+@end example
+
+Although Rust libraries are not publicly exported, we can still select them via
+the Guix command-line interface via a Guile expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+To unbundle most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-curl-sys-0.4.80+curl-8.12.1
+  (crate-source "curl-sys" "0.4.80+curl-8.12.1"
+                "0d7ppx4kq77hc5nyff6jydmfabpgd0i3ppjvn8x0q833mhpdzxsm"
+                #:snippet '(delete-file-recursively "curl")))
+@end lisp
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+  (crate-source "bzip2-sys" "0.1.13+1.0.8"
+                "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+                #:snippet
+                '(begin
+                   (delete-file-recursively "bzip2-1.0.8")
+                   (delete-file "build.rs")
+                   (with-output-to-file "build.rs"
+                     (lambda _
+                       (format #t "fn main() @{~@@
+                        println!(\"cargo:rustc-link-lib=bz2\");~@@
+                        @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires a build process
+that involves other packages, we should make a full package in @code{(gnu
+packages rust-sources)} first and reference it in the imported definition.
+
+For example, we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to reference it.
+
+@lisp
+(define rust-ring-0.17.14 rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+
+To facilitate various tasks in the common workflow, several scripts are provided
+in the @file{etc/teams/rust} directory of Guix source tree.
+@end enumerate
+
+@node Cargo Workspaces and Development Snapshots
+@subsubsection Cargo Workspaces and Development Snapshots
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+  (package
+   (name "niri")
+   (version "25.02")
+   (source (origin
+             (method git-fetch)
+             (uri (git-reference
+                   (url "https://github.com/YaLTeR/niri")
+                   (commit (string-append "v" version))))
+             (file-name (git-file-name name version))
+             (sha256
+              (base32
+               "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+   (build-system cargo-build-system)
+   (arguments
+    (list #:install-source? #f
+          #:phases
+          #~(modify-phases %standard-phases
+              (add-after 'unpack 'use-guix-vendored-dependencies
+                (lambda _
+                  (substitute* "Cargo.toml"
+                    (("# version =.*")
+                     "version = \"*\"")
+                    (("git.*optional")
+                     "version = \"*\", optional")
+                    (("^git = .*")
+                     "")))))))
+   (native-inputs
+    (list pkg-config))
+   (inputs
+    (cons* clang
+           libdisplay-info
+           libinput-minimal
+           libseat
+           libxkbcommon
+           mesa
+           pango
+           pipewire
+           wayland
+           (cargo-inputs 'niri)))
+   (home-page "https://github.com/YaLTeR/niri")
+   (synopsis "Scrollable-tiling Wayland compositor")
+   (description
+    "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format.  It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+   (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies.  When packaging a Cargo
+workspace dependency, parameter @code{#:cargo-package-crates} is
+required.
+
+@lisp
+(define-public rust-pipewire-0.8.0.fd3d8f7
+  (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+        (revision "0"))
+    (package
+      (name "rust-pipewire")
+      (version (git-version "0.8.0" revision commit))
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+               (commit commit)))
+         (file-name (git-file-name name version))
+         (sha256
+          (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+      (build-system cargo-build-system)
+      (arguments
+       (list #:skip-build? #t
+             #:cargo-package-crates
+             ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+      (inputs (cargo-inputs 'rust-pipewire-0.8.0.fd3d8f7))
+      (home-page "https://pipewire.org/")
+      (synopsis "Rust bindings for PipeWire")
+      (description "This package provides Rust bindings for PipeWire.")
+      (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 rust-pipewire-0.8.0.fd3d8f7)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 rust-pipewire-0.8.0.fd3d8f7)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 rust-pipewire-0.8.0.fd3d8f7)
+(define rust-libspa-sys-0.8.0.fd3d8f7 rust-pipewire-0.8.0.fd3d8f7)
+@end lisp
+
+@node Using Rust Libraries in Other Build Systems
+@subsubsection Using Rust Libraries in Other Build Systems
+
+In this example, we'll package @code{libchewing}, which combines two build
+systems.
+
+When building Rust packages in other build systems, we need to add @code{rust},
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+For cross-compilation support, we'll also add @code{rust-sysroot} created with
+@code{(make-rust-sysroot (%current-target-system))} to @code{native-inputs}, and
+set @code{#:cargo-target} for @code{cargo-build-system}'s build phases.
+
+@lisp
+(define-public libchewing
+  (package
+    (name "libchewing")
+    (version "0.9.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/chewing/libchewing")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0gh64wvrk5pn0fhmpvj1j99d5g7f7697rk96zbkc8l72yjr819z5"))))
+    (build-system cmake-build-system)
+    (arguments
+     (list #:imported-modules
+           (append %cmake-build-system-modules
+                   %cargo-build-system-modules)
+           #:modules
+           '(((guix build cargo-build-system) #:prefix cargo:)
+             (guix build cmake-build-system)
+             (guix build utils))
+           #:phases
+           #~(modify-phases %standard-phases
+               (add-after 'unpack 'prepare-cargo-build-system
+                 (lambda args
+                   (for-each
+                    (lambda (phase)
+                      (format #t "Running cargo phase: ~a~%" phase)
+                      (apply (assoc-ref cargo:%standard-phases phase)
+                             ;; For cross-compilation.
+                             #:cargo-target #$(cargo-triplet)
+                             args))
+                    '(unpack-rust-crates
+                      configure
+                      check-for-pregenerated-files
+                      patch-cargo-checksums)))))))
+    (native-inputs
+     (append
+      (list rust `(,rust "cargo") )
+      ;; For cross-compilation.
+      (or (and=> (%current-target-system)
+                 (compose list make-rust-sysroot))
+          '())))
+    (inputs
+     (cons* corrosion ncurses sqlite (cargo-inputs 'libchewing)))
+    (synopsis "Chinese phonetic input method")
+    (description "Chewing is an intelligent phonetic (Zhuyin/Bopomofo) input
+method, one of the most popular choices for Traditional Chinese users.")
+    (home-page "https://chewing.im/")
+    (license license:lgpl2.1+)))
+@end lisp
+
+
 @c *********************************************************************
 @node System Configuration
 @chapter System Configuration

doc/guix.texi

@@ -9573,12 +9573,13 @@ crate.  Unless @code{install-source? #f} is defined it will also install a
 source crate repository of itself and unpacked sources, to ease in future
 hacking on Rust packages.
 
-This build system supports Cargo workspaces.  Parameter
+This build system supports
-@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+@url{https://doc.rust-lang.org/cargo/reference/workspaces.html, Cargo
-library crates to package in the @code{package} phase.  Specified crates are
+workspaces}.  Parameter @code{#:cargo-package-crates} (default: @code{''()})
-packaged from left to right, in case there's dependency among them.  For
+allows specifying names of library crates to package in the @code{package}
-example, specifying @code{''("pcre2-sys" "pcre2")} will package
+phase.  Specified crates are packaged from left to right, in case there's
-@code{"pcre2-sys"} first and then @code{"pcre2"}.  Parameter
+dependency among them.  For example, specifying @code{''("pcre2-sys" "pcre2")}
+will package @code{"pcre2-sys"} first and then @code{"pcre2"}.  Parameter
 @code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
 binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
 for example.

gnu/packages/rust-crates.scm

@@ -27,10 +27,6 @@
 
 ;;;
 ;;; This file is managed by ‘guix import’.  Do NOT add definitions manually.
-;;;
-;;; Commit changes to: https://codeberg.org/guix/guix-rust-registry
-;;;
-
 ;;;
 ;;; Rust libraries fetched from crates.io and non-workspace development
 ;;; snapshots.

gnu/packages/rust-sources.scm

@@ -36,10 +36,6 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages textutils))
 
-;;;
-;;; Commit changes to: https://codeberg.org/guix/guix-rust-registry
-;;;
-
 ;;;
 ;;; Cargo workspaces and Rust libraries requiring external inputs to unbundle.
 ;;; These packages are hidden, as they are not interesting to users.