2

Background

When I've added the first Overlay for Nixpkgs, I found out that a bunch of system utils got built for a different version:

these derivations will be built:
  /nix/store/028dqnwq36xja16gba3gckq5mcprpn06-postfix-main.cf.drv
  /nix/store/b2sch2538ny2drdf9zzknf38grn8d8r3-pcre-8.42.drv
  /nix/store/i1k9ksk32ca441zap40z3zddy7bhqx3n-zlib-1.2.11.drv
  /nix/store/sypawsb3cwqnnhdl1barv2d8nyvbsxyv-coreutils-8.29.drv
  /nix/store/xa4vnajxck2zgvjwp7l71lm11hqnz32r-findutils-4.6.0.drv
...

which is time and space consuming. I tried to figure out what's going on then end up with this question.

Summary

The idea of self and super of overlays is that self is the accumulated result after all overlays have applied, while super is the result of applying the previous overlay.

I thought that an attribute that have not been touched by will be the same between self and super, but some are not.

let
  nixpkgs = import <nixpkgs> {
    overlays = [
      # This overlay obtains self.bash and super.bash, and save them under the
      # attrs "bash-from-self" and "bash-from-super" for further examination
      (self: super: {
        bash-from-self = self.bash;
        bash-from-super = super.bash;
      })
    ];
  };
  # Retrieve bash-from-self (self.bash) and bash-from-super (super.bash)
  # from the overlayed nixpkgs
  inherit (nixpkgs) bash-from-self bash-from-super;
in {
  # Check if bash-from-self (self.bash) and bash-from-super (super.bash)
  # are same
  isBashSame = (bash-from-self == bash-from-super);
  inherit bash-from-self bash-from-super;
}

The above evaluates to:

{ isBashSame = false; 
  bash-from-self = «derivation /nix/store/zvy7mbpxqlplqpflqn5xk9szx25s4mhg-bash-4.4-p23.drv»;
  bash-from-super = «derivation /nix/store/2i91sj16snsphvjrbsa62z8m4zhs261c-bash-4.4-p23.drv»; }

Showing that self.bash and super.bash is not the same, even that the bash attribute isn't touched in any overlays. Why is this happening, or did have some concepts missing in mind?


Details

More different attributes

Besides of bash, there're more attributes that are different:

let
  isAttrSame =
    attrName:
    let
      nixpkgs = import <nixpkgs> {
        overlays = [
          (_self: _super: { inherit _self _super; })
        ];
      };
      self = nixpkgs._self."${attrName}";
      super = nixpkgs._super."${attrName}";
      isSame = self == super;
    in
      isSame
  ;
in {
  coreutils = isAttrSame "coreutils";
  bash = isAttrSame "bash";
  zsh = isAttrSame "zsh";
  zlib = isAttrSame "zlib";
  stdenv = isAttrSame "stdenv";
  findutils = isAttrSame "findutils";
  gnutar = isAttrSame "gnutar";
  gcc = isAttrSame "gcc";
}
{
  bash = false;
  coreutils = false;
  findutils = false;
  gcc = true;
  gnutar = true;
  stdenv = true;
  zlib = false;
  zsh = true;
}

The builder of self.bash is super.bash?

let
  nixpkgs = import <nixpkgs> {
    overlays = [
      (self: super: {
        bash-from-self = self.bash;
        bash-from-super = super.bash;
      })
    ];
  };
  inherit (nixpkgs) bash-from-self bash-from-super;
in {
  bash-from-self-builder = bash-from-self.drvAttrs.builder;
  bash-from-super-builder = bash-from-super.drvAttrs.builder;
  bash-from-self-outPath = bash-from-self.outPath;
  bash-from-super-outPath = bash-from-super.outPath;
}
{ bash-from-self-builder = "/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools/bin/bash";
  bash-from-self-outPath = "/nix/store/06z61jbgs0vkw4i9cqqf9yl7zsfkkhw2-bash-4.4-p23";
  bash-from-super-builder = "/nix/store/06z61jbgs0vkw4i9cqqf9yl7zsfkkhw2-bash-4.4-p23/bin/bash";
  bash-from-super-outPath = "/nix/store/2avim7j13k75k26w18g6br8gai869nm9-bash-4.4-p23"; }

bash-from-self-outPath is the bash-from-super-builder (06z61...khw2-bash-4.4-p23).

So super.bash uses self.bash to build itself, producing another bash (2avim...9nm9-bash-4.4-p23)?

bootstrap-tools/bash --builds--> self.bash --builds--> super.bash

Why this is a problem

I want some of my packages defined in an overlay to depend on bash, coreutils and stuff. I want to use the original version of them provided directly from <nixpkgs>, not those may be overwritten by later overlays. So, in this case, seems that I should choose super.* instead of self.* to be the dependencies.

But some super.stuff is not the original nixpkgs.stuff, this causes rebuilding them as there's no binary cache and is a waste of disk space. And self.stuff can probably be overwritten by later overlays. What can I do?

zetavg
  • 188
  • 8
  • 1
    It seems to have to do with stdenvOverrides, https://github.com/NixOS/nixpkgs/blob/ea863cd334be044aee28e77ff4446f16063ec30e/pkgs/top-level/stage.nix#L190 – Robert Hensing Apr 06 '19 at 15:16

0 Answers0