using provided RNG seed: reconfigurator-cli-test
> # Load an example system. The sled with serial5 is marked non-provisionable
> # so that discretionary zones don't make their way onto it. (We're going to
> # expunge it below to test that we don't try and update zone image sources
> # on expunged sleds.)
> load-example --nsleds 7 --ndisks-per-sled 1 --sled-policy 5:non-provisionable
loaded example system with:
- collection: f45ba181-4b56-42cc-a762-874d90184a43
- blueprint: dbcbd3d6-41ff-48ae-ac0b-1becc9b2fd21


> sled-list
ID                                   SERIAL  NZPOOLS SUBNET                  
2b8f0cb3-0295-4b3c-bc58-4fe88b57112c serial1 1       fd00:1122:3344:102::/64 
98e6b7c2-2efa-41ca-b20a-0a4d61102fe6 serial0 1       fd00:1122:3344:101::/64 
9a867dc9-d505-427f-9eff-cdb1d4d9bd73 serial5 1       fd00:1122:3344:106::/64 
aff6c093-197d-42c5-ad80-9f10ba051a34 serial3 1       fd00:1122:3344:104::/64 
b82ede02-399c-48c6-a1de-411df4fa49a7 serial4 1       fd00:1122:3344:105::/64 
d81c6a84-79b8-4958-ae41-ea46c9b19763 serial2 1       fd00:1122:3344:103::/64 
e96e226f-4ed9-4c01-91b9-69a9cd076c9e serial6 1       fd00:1122:3344:107::/64 


> # Create a TUF repository from a fake manifest. (The output TUF repo is
> # written to a temporary directory that this invocation of `reconfigurator-cli`
> # is running out of as its working directory.)
> generate-fake-repo 1.0.0
created repo-1.0.0.zip for system version 1.0.0

> # Create a second TUF repository from a different fake manifest.
> generate-fake-repo 2.0.0
created repo-2.0.0.zip for system version 2.0.0


> # Load the target release from the first TUF repository.
> set target-release repo-1.0.0.zip
set target release based on repo-1.0.0.zip


> # On one sled, update the install dataset.
> sled-update-install-dataset serial0 --to-target-release
sled 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6: install dataset updated: to target release (system version 1.0.0)


> # On another sled, simulate an error reading the zone manifest.
> sled-update-install-dataset serial1 --with-manifest-error
sled 2b8f0cb3-0295-4b3c-bc58-4fe88b57112c: install dataset updated: simulated error obtaining zone manifest


> # On a third sled, update the install dataset and simulate a mupdate override.
> # Also set it in the blueprint -- this simulates the situation where the mupdate
> # override is in progress and will be cleared in the future.
> sled-update-install-dataset serial2 --to-target-release
sled d81c6a84-79b8-4958-ae41-ea46c9b19763: install dataset updated: to target release (system version 1.0.0)

> sled-set serial2 mupdate-override ffffffff-ffff-ffff-ffff-ffffffffffff
set sled d81c6a84-79b8-4958-ae41-ea46c9b19763 mupdate override: unset -> ffffffff-ffff-ffff-ffff-ffffffffffff

> blueprint-edit latest set-remove-mupdate-override serial2 ffffffff-ffff-ffff-ffff-ffffffffffff
blueprint 8da82a8e-bf97-4fbd-8ddd-9f6462732cf1 created from latest blueprint (dbcbd3d6-41ff-48ae-ac0b-1becc9b2fd21): set remove_mupdate_override to ffffffff-ffff-ffff-ffff-ffffffffffff

> # Apply the latest blueprint to serial2's inventory so that the noop check
> # exercises the mupdate-override-set-in-blueprint path rather than firing
> # inventory-stale on the bumped sled-agent generation.
> sled-set serial2 omicron-config latest
set sled d81c6a84-79b8-4958-ae41-ea46c9b19763 omicron config from latest blueprint (8da82a8e-bf97-4fbd-8ddd-9f6462732cf1)


> # On a fourth sled, simulate an error validating the install dataset image on one zone.
> # We pick ntp because internal-ntp is non-discretionary.
> sled-update-install-dataset serial3 --to-target-release --with-zone-error ntp
sled aff6c093-197d-42c5-ad80-9f10ba051a34: install dataset updated: to target release (system version 1.0.0, 1 zone errors)


> # On a fifth sled, set the install dataset to the repo-2.0.0.zip generated by the
> # second TUF repository.
> sled-update-install-dataset serial4 --from-repo repo-2.0.0.zip
sled b82ede02-399c-48c6-a1de-411df4fa49a7: install dataset updated: from repo at repo-2.0.0.zip (system version 2.0.0)


> # On the sixth sled, update to the target release (so it shows up in inventory).
> # Then, mark the sled expunged (in the planning input).
> sled-update-install-dataset serial5 --to-target-release
sled 9a867dc9-d505-427f-9eff-cdb1d4d9bd73: install dataset updated: to target release (system version 1.0.0)

> sled-set serial5 policy expunged
set sled 9a867dc9-d505-427f-9eff-cdb1d4d9bd73 policy to expunged


> # On the seventh sled, update to the target release but hide the sled
> # from inventory. This should prevent changes to the blueprint for
> # this sled.
> sled-update-install-dataset serial6 --to-target-release
sled e96e226f-4ed9-4c01-91b9-69a9cd076c9e: install dataset updated: to target release (system version 1.0.0)

> sled-set serial6 inventory-hidden
set sled e96e226f-4ed9-4c01-91b9-69a9cd076c9e inventory visibility: visible -> hidden


> # Generate an inventory and run a blueprint planning step.
> inventory-generate
generated inventory collection eb0796d5-ab8a-4f7b-a884-b4aeacb8ab51 from configured sleds

> blueprint-plan latest latest
WARN skipping zones eligible for cleanup check (sled not present in latest inventory collection), sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e
WARN no inventory found for in-service sled, phase: do_plan_mupdate_override, sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e
WARN skipped noop image source check on sled, sled_id: 2b8f0cb3-0295-4b3c-bc58-4fe88b57112c, reason: error retrieving zone manifest: reconfigurator-sim simulated error: simulated error obtaining zone manifest
INFO performed noop zone image source checks on sled, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6, num_total: 6, num_already_artifact: 0, num_eligible: 6, num_ineligible: 0
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6
INFO performed noop zone image source checks on sled, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, num_total: 6, num_already_artifact: 0, num_eligible: 5, num_ineligible: 1
WARN zone manifest inventory indicated install dataset artifact is invalid, not using artifact (this is abnormal), sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, zone_id: e8fe709c-725f-4bb2-b714-ffcda13a9e54, kind: internal_ntp, file_name: ntp.tar.gz, message: reconfigurator-sim: simulated error validating zone image
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34
INFO performed noop zone image source checks on sled, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, num_total: 2, num_already_artifact: 0, num_eligible: 0, num_ineligible: 2
INFO install dataset artifact hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, zone_id: b910534b-2a53-4335-a3d9-5311d2f3186a, kind: internal_ntp, file_name: ntp.tar.gz, expected_hash: 8f0613ee91aac7c8c25fb0abfd94f7b768e44e1bdb683663d3eb6222b5318b8e
INFO install dataset artifact hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, zone_id: ecbe0b3d-1acc-44b2-b6d4-f4d2770516e4, kind: crucible, file_name: crucible.tar.gz, expected_hash: e7de2c66e29bebe8288deda473918e827b4755abe82bb7859630eba9ef8a5736
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7
INFO skipped noop image source check on sled, sled_id: d81c6a84-79b8-4958-ae41-ea46c9b19763, reason: remove_mupdate_override is set in the blueprint (ffffffff-ffff-ffff-ffff-ffffffffffff)
INFO skipped noop image source check on sled, sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e, reason: sled not found in inventory
generated blueprint 58d5e830-0884-47d8-a7cd-b2b3751adeb4 based on parent blueprint 8da82a8e-bf97-4fbd-8ddd-9f6462732cf1
blueprint source: planner with report:
planning report:
* noop converting 6/6 install-dataset zones to artifact store on sled 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6
* noop converting 5/6 install-dataset zones to artifact store on sled aff6c093-197d-42c5-ad80-9f10ba051a34
* zone adds waiting on blockers
* zone adds and updates are blocked:
  - sleds have remove mupdate override set in blueprint: d81c6a84-79b8-4958-ae41-ea46c9b19763
  - sleds have deployment units with image sources not set to Artifact:
    - sled 2b8f0cb3-0295-4b3c-bc58-4fe88b57112c: 7 zones
    - sled aff6c093-197d-42c5-ad80-9f10ba051a34: 1 zone
    - sled b82ede02-399c-48c6-a1de-411df4fa49a7: 2 zones
    - sled d81c6a84-79b8-4958-ae41-ea46c9b19763: 2 zones
    - sled e96e226f-4ed9-4c01-91b9-69a9cd076c9e: 2 zones

* zone updates waiting on zone add blockers
* waiting to update top-level nexus_generation: some non-Nexus zone are not yet updated
Measurement updates:
Waiting on zone add/update blockers




> # This diff should show expected changes to the blueprint.
> blueprint-diff latest
from: blueprint 8da82a8e-bf97-4fbd-8ddd-9f6462732cf1
to:   blueprint 58d5e830-0884-47d8-a7cd-b2b3751adeb4

 MODIFIED SLEDS:

  sled 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6 (active, config generation 2 -> 3):

    measurements:
    ---------------------------
    hash      version          
    ---------------------------
    unknown   (unknown version)


    host phase 2 contents:
    ------------------------
    slot   boot image source
    ------------------------
    A      current contents 
    B      current contents 


    physical disks:
    ------------------------------------------------------------------------------------
    vendor        model        serial                                        disposition
    ------------------------------------------------------------------------------------
    fake-vendor   fake-model   serial-c6d33b64-fb96-4129-bab1-7878a06a5f9b   in service 


    datasets:
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset id                             disposition   quota     reservation   compression
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crucible                                                              42430c80-7836-4191-a4f6-bcee749010fe   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/external_dns                                                    ad41be71-6c15-4428-b510-20ceacde4fa6   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/internal_dns                                                    a4c3032e-21fa-4d4a-b040-a7e3c572cf3c   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone                                                            4617d206-4330-4dfa-b9f3-f63a3db834f9   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone/oxz_crucible_5199c033-4cf9-4ab6-8ae7-566bd7606363          4847a96e-a267-4ae7-aa3d-805c1e77f81e   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone/oxz_crucible_pantry_ba4994a8-23f9-4b1a-a84f-a08d74591389   43931274-7fe8-4077-825d-dff2bc8efa58   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone/oxz_external_dns_803bfb63-c246-41db-b0da-d3b87ddfc63d      1bca7f71-5e42-4749-91ec-fa40793a3a9a   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone/oxz_internal_dns_427ec88f-f467-42fa-9bbb-66a91a36103c      3ac089c9-9dec-465b-863a-188e80d71fb4   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone/oxz_nexus_0c71b3b2-6ceb-4e8f-b020-b08675e83038             4f60b534-eaa3-40a1-b60f-bfdf147af478   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/zone/oxz_ntp_6444f8a5-6465-4f0b-a549-1993c113569c               686c19cf-a0d7-45f6-866f-c564612b2664   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/debug                                                           248c6c10-1ac6-45de-bb55-ede36ca56bbd   in service    100 GiB   none          gzip-9     
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/crypt/local_storage                                                   cdf3684f-a6cf-4449-b9ec-e696b2c663e2   in service    none      none          off        
    oxp_c6d33b64-fb96-4129-bab1-7878a06a5f9b/local_storage_unencrypted                                             793ac181-1b01-403c-850d-7f5c54bda6c9   in service    none      none          off        


    omicron zones:
    ------------------------------------------------------------------------------------------------------------------------
    zone type         zone id                                image source                disposition   underlay IP          
    ------------------------------------------------------------------------------------------------------------------------
*   crucible          5199c033-4cf9-4ab6-8ae7-566bd7606363   - install dataset           in service    fd00:1122:3344:101::7
     └─                                                      + artifact: version 1.0.0                                      
*   crucible_pantry   ba4994a8-23f9-4b1a-a84f-a08d74591389   - install dataset           in service    fd00:1122:3344:101::6
     └─                                                      + artifact: version 1.0.0                                      
*   external_dns      803bfb63-c246-41db-b0da-d3b87ddfc63d   - install dataset           in service    fd00:1122:3344:101::5
     └─                                                      + artifact: version 1.0.0                                      
*   internal_dns      427ec88f-f467-42fa-9bbb-66a91a36103c   - install dataset           in service    fd00:1122:3344:2::1  
     └─                                                      + artifact: version 1.0.0                                      
*   internal_ntp      6444f8a5-6465-4f0b-a549-1993c113569c   - install dataset           in service    fd00:1122:3344:101::3
     └─                                                      + artifact: version 1.0.0                                      
*   nexus             0c71b3b2-6ceb-4e8f-b020-b08675e83038   - install dataset           in service    fd00:1122:3344:101::4
     └─                                                      + artifact: version 1.0.0                                      


  sled 9a867dc9-d505-427f-9eff-cdb1d4d9bd73 (active -> decommissioned, config generation 2 -> 3):

    measurements:
    ---------------------------
    hash      version          
    ---------------------------
    unknown   (unknown version)


    host phase 2 contents:
    ------------------------
    slot   boot image source
    ------------------------
    A      current contents 
    B      current contents 


    physical disks:
    ---------------------------------------------------------------------------------------
    vendor        model        serial                                        disposition   
    ---------------------------------------------------------------------------------------
*   fake-vendor   fake-model   serial-5ad21c90-792e-445b-b25e-285723c44243   - in service  
     └─                                                                      + expunged ✓  


    datasets:
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                            dataset id                             disposition    quota     reservation   compression
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/crucible                                                       03652e2c-d957-4533-9464-a8c5472ab672   - in service   none      none          off        
     └─                                                                                                                                            + expunged                                        
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/crypt/zone                                                     f9c25bcc-cd19-4003-a23a-807ae76e35aa   - in service   none      none          off        
     └─                                                                                                                                            + expunged                                        
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/crypt/zone/oxz_crucible_824150c0-3fa4-4bac-9d14-c47ad04c9f3a   e0af3fa7-6e9e-4eec-80a4-c406858a390a   - in service   none      none          off        
     └─                                                                                                                                            + expunged                                        
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/crypt/zone/oxz_ntp_db288a1e-c33c-44ca-8c79-9a8978afa34d        7fd50c89-afc0-4a37-ac93-ba5273120d32   - in service   none      none          off        
     └─                                                                                                                                            + expunged                                        
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/crypt/debug                                                    639da9cf-2ce6-4e90-b95d-a0d0fc57d0f2   - in service   100 GiB   none          gzip-9     
     └─                                                                                                                                            + expunged                                        
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/crypt/local_storage                                            51086d20-09ec-454c-9127-435beba2b8f0   - in service   none      none          off        
     └─                                                                                                                                            + expunged                                        
*   oxp_5ad21c90-792e-445b-b25e-285723c44243/local_storage_unencrypted                                      e36ae01d-7975-429c-b802-f1bbc3b032b3   - in service   none      none          off        
     └─                                                                                                                                            + expunged                                        


    omicron zones:
    --------------------------------------------------------------------------------------------------------------
    zone type      zone id                                image source      disposition      underlay IP          
    --------------------------------------------------------------------------------------------------------------
*   crucible       824150c0-3fa4-4bac-9d14-c47ad04c9f3a   install dataset   - in service     fd00:1122:3344:106::4
     └─                                                                     + expunged ✓                          
*   internal_ntp   db288a1e-c33c-44ca-8c79-9a8978afa34d   install dataset   - in service     fd00:1122:3344:106::3
     └─                                                                     + expunged ✓                          


  sled aff6c093-197d-42c5-ad80-9f10ba051a34 (active, config generation 2 -> 3):

    measurements:
    ---------------------------
    hash      version          
    ---------------------------
    unknown   (unknown version)


    host phase 2 contents:
    ------------------------
    slot   boot image source
    ------------------------
    A      current contents 
    B      current contents 


    physical disks:
    ------------------------------------------------------------------------------------
    vendor        model        serial                                        disposition
    ------------------------------------------------------------------------------------
    fake-vendor   fake-model   serial-f3e1cbbc-5682-46ce-93a2-00a4a603bb63   in service 


    datasets:
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset id                             disposition   quota     reservation   compression
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crucible                                                              7cd9d655-5520-4c1b-bd38-2f510970b252   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/external_dns                                                    24bc6a37-a936-4453-96fe-61f15e9535d7   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/internal_dns                                                    0e924845-9e93-4313-bcce-23d534fcc633   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone                                                            cbb667a8-71f0-4ef3-a8ec-3149eaab45e0   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone/oxz_crucible_8e3dd7a4-75a3-4917-a6f4-0991bbdef7ea          79712d30-0421-4883-859c-a9b3da4fde2f   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone/oxz_crucible_pantry_d07a1fed-4235-4821-a1e5-f7eb2646ff33   aa71e3e0-8b92-402e-89a9-4f3252af2863   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone/oxz_external_dns_43a0588f-5b57-469b-a173-db6cb6105e4c      a2807343-8c57-45f2-877a-ca064e2e28e2   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone/oxz_internal_dns_97753dbd-5a0f-4273-b1be-db6bb2b69381      27521fba-e5e3-418f-975b-a499010bf840   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone/oxz_nexus_33862f97-2897-4d53-a9a6-78a80f7eb13f             2f73afe1-14e7-4625-85a6-645d9efaa370   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/zone/oxz_ntp_e8fe709c-725f-4bb2-b714-ffcda13a9e54               f41fb7c9-c08d-463b-9095-dd7b7a39bd70   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/debug                                                           d6f0564e-e4a8-4f6f-b122-0314ff473b20   in service    100 GiB   none          gzip-9     
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/crypt/local_storage                                                   febe87b8-c8ac-4401-b9ee-bbc3be700946   in service    none      none          off        
    oxp_f3e1cbbc-5682-46ce-93a2-00a4a603bb63/local_storage_unencrypted                                             b8457107-50ca-4454-a17d-f9c0d4f94cde   in service    none      none          off        


    omicron zones:
    ------------------------------------------------------------------------------------------------------------------------
    zone type         zone id                                image source                disposition   underlay IP          
    ------------------------------------------------------------------------------------------------------------------------
    internal_ntp      e8fe709c-725f-4bb2-b714-ffcda13a9e54   install dataset             in service    fd00:1122:3344:104::3
*   crucible          8e3dd7a4-75a3-4917-a6f4-0991bbdef7ea   - install dataset           in service    fd00:1122:3344:104::7
     └─                                                      + artifact: version 1.0.0                                      
*   crucible_pantry   d07a1fed-4235-4821-a1e5-f7eb2646ff33   - install dataset           in service    fd00:1122:3344:104::6
     └─                                                      + artifact: version 1.0.0                                      
*   external_dns      43a0588f-5b57-469b-a173-db6cb6105e4c   - install dataset           in service    fd00:1122:3344:104::5
     └─                                                      + artifact: version 1.0.0                                      
*   internal_dns      97753dbd-5a0f-4273-b1be-db6bb2b69381   - install dataset           in service    fd00:1122:3344:3::1  
     └─                                                      + artifact: version 1.0.0                                      
*   nexus             33862f97-2897-4d53-a9a6-78a80f7eb13f   - install dataset           in service    fd00:1122:3344:104::4
     └─                                                      + artifact: version 1.0.0                                      


 COCKROACHDB SETTINGS:
    state fingerprint:::::::::::::::::   (none) (unchanged)
    cluster.preserve_downgrade_option:   (do not modify) (unchanged)

 METADATA:
    internal DNS version:::   1 (unchanged)
    external DNS version:::   1 (unchanged)
    target release min gen:   1 (unchanged)
    nexus gen::::::::::::::   1 (unchanged)

 OXIMETER SETTINGS:
    generation:   1 (unchanged)
    read from::   SingleNode (unchanged)


internal DNS:
* DNS zone: "control-plane.oxide.internal": 
-   name: 824150c0-3fa4-4bac-9d14-c47ad04c9f3a.host          (records: 1)
-       AAAA fd00:1122:3344:106::4
-   name: _crucible._tcp.824150c0-3fa4-4bac-9d14-c47ad04c9f3a (records: 1)
-       SRV  port 32345 824150c0-3fa4-4bac-9d14-c47ad04c9f3a.host.control-plane.oxide.internal
*   name: _internal-ntp._tcp                                 (records: 7 -> 6)
-       SRV  port   123 62620961-fc4a-481e-968b-f5acbac0dc63.host.control-plane.oxide.internal
-       SRV  port   123 6444f8a5-6465-4f0b-a549-1993c113569c.host.control-plane.oxide.internal
-       SRV  port   123 b910534b-2a53-4335-a3d9-5311d2f3186a.host.control-plane.oxide.internal
-       SRV  port   123 c800ba17-240e-4b72-8ae6-afc30b6baa96.host.control-plane.oxide.internal
-       SRV  port   123 db288a1e-c33c-44ca-8c79-9a8978afa34d.host.control-plane.oxide.internal
-       SRV  port   123 dd66f033-4fe8-438e-afb4-29d3561d4c3e.host.control-plane.oxide.internal
-       SRV  port   123 e8fe709c-725f-4bb2-b714-ffcda13a9e54.host.control-plane.oxide.internal
+       SRV  port   123 62620961-fc4a-481e-968b-f5acbac0dc63.host.control-plane.oxide.internal
+       SRV  port   123 6444f8a5-6465-4f0b-a549-1993c113569c.host.control-plane.oxide.internal
+       SRV  port   123 b910534b-2a53-4335-a3d9-5311d2f3186a.host.control-plane.oxide.internal
+       SRV  port   123 c800ba17-240e-4b72-8ae6-afc30b6baa96.host.control-plane.oxide.internal
+       SRV  port   123 dd66f033-4fe8-438e-afb4-29d3561d4c3e.host.control-plane.oxide.internal
+       SRV  port   123 e8fe709c-725f-4bb2-b714-ffcda13a9e54.host.control-plane.oxide.internal
-   name: db288a1e-c33c-44ca-8c79-9a8978afa34d.host          (records: 1)
-       AAAA fd00:1122:3344:106::3
    unchanged names: 51 (records: 68)

external DNS:
  DNS zone: "oxide.example" (unchanged)
    unchanged names: 5 (records: 9)




> # Bring the hidden sled back.
> sled-set serial6 inventory-visible
set sled e96e226f-4ed9-4c01-91b9-69a9cd076c9e inventory visibility: hidden -> visible


> # The first plan bumped the sled-agent generation on the sleds where it did
> # noop conversions (serial0, serial3) and where it set remove_mupdate_override
> # (serial2). Apply that blueprint to those sleds' inventories so the next
> # plan's noop check doesn't fire the inventory-stale path for them.
> sled-set serial0 omicron-config latest
set sled 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6 omicron config from latest blueprint (58d5e830-0884-47d8-a7cd-b2b3751adeb4)

> sled-set serial2 omicron-config latest
set sled d81c6a84-79b8-4958-ae41-ea46c9b19763 omicron config from latest blueprint (58d5e830-0884-47d8-a7cd-b2b3751adeb4)

> sled-set serial3 omicron-config latest
set sled aff6c093-197d-42c5-ad80-9f10ba051a34 omicron config from latest blueprint (58d5e830-0884-47d8-a7cd-b2b3751adeb4)


> # Run another inventory and blueprint planning step.
> inventory-generate
generated inventory collection 61f451b3-2121-4ed6-91c7-a550054f6c21 from configured sleds

> blueprint-plan latest latest
WARN skipped noop image source check on sled, sled_id: 2b8f0cb3-0295-4b3c-bc58-4fe88b57112c, reason: error retrieving zone manifest: reconfigurator-sim simulated error: simulated error obtaining zone manifest
INFO performed noop zone image source checks on sled, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6, num_total: 6, num_already_artifact: 6, num_eligible: 0, num_ineligible: 0
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: 98e6b7c2-2efa-41ca-b20a-0a4d61102fe6
INFO performed noop zone image source checks on sled, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, num_total: 6, num_already_artifact: 5, num_eligible: 0, num_ineligible: 1
WARN zone manifest inventory indicated install dataset artifact is invalid, not using artifact (this is abnormal), sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, zone_id: e8fe709c-725f-4bb2-b714-ffcda13a9e54, kind: internal_ntp, file_name: ntp.tar.gz, message: reconfigurator-sim: simulated error validating zone image
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: aff6c093-197d-42c5-ad80-9f10ba051a34
INFO performed noop zone image source checks on sled, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, num_total: 2, num_already_artifact: 0, num_eligible: 0, num_ineligible: 2
INFO install dataset artifact hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, zone_id: b910534b-2a53-4335-a3d9-5311d2f3186a, kind: internal_ntp, file_name: ntp.tar.gz, expected_hash: 8f0613ee91aac7c8c25fb0abfd94f7b768e44e1bdb683663d3eb6222b5318b8e
INFO install dataset artifact hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, zone_id: ecbe0b3d-1acc-44b2-b6d4-f4d2770516e4, kind: crucible, file_name: crucible.tar.gz, expected_hash: e7de2c66e29bebe8288deda473918e827b4755abe82bb7859630eba9ef8a5736
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: b82ede02-399c-48c6-a1de-411df4fa49a7
INFO skipped noop image source check on sled, sled_id: d81c6a84-79b8-4958-ae41-ea46c9b19763, reason: remove_mupdate_override is set in the blueprint (ffffffff-ffff-ffff-ffff-ffffffffffff)
INFO performed noop zone image source checks on sled, sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e, num_total: 2, num_already_artifact: 0, num_eligible: 2, num_ineligible: 0
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e, slot: a, expected_hash: 0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
INFO BootPartitionDetails inventory hash not found in TUF repo, ignoring for noop checks, sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e, slot: b, expected_hash: 0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b
INFO current sled measurements are in an unknown state, sled_id: e96e226f-4ed9-4c01-91b9-69a9cd076c9e
generated blueprint af934083-59b5-4bf6-8966-6fb5292c29e1 based on parent blueprint 58d5e830-0884-47d8-a7cd-b2b3751adeb4
blueprint source: planner with report:
planning report:
* noop converting 2/2 install-dataset zones to artifact store on sled e96e226f-4ed9-4c01-91b9-69a9cd076c9e
* zone adds waiting on blockers
* zone adds and updates are blocked:
  - sleds have remove mupdate override set in blueprint: d81c6a84-79b8-4958-ae41-ea46c9b19763
  - sleds have deployment units with image sources not set to Artifact:
    - sled 2b8f0cb3-0295-4b3c-bc58-4fe88b57112c: 7 zones
    - sled aff6c093-197d-42c5-ad80-9f10ba051a34: 1 zone
    - sled b82ede02-399c-48c6-a1de-411df4fa49a7: 2 zones
    - sled d81c6a84-79b8-4958-ae41-ea46c9b19763: 2 zones

* zone updates waiting on zone add blockers
* waiting to update top-level nexus_generation: some non-Nexus zone are not yet updated
Measurement updates:
Waiting on zone add/update blockers




> # This diff should show changes to the sled that's back in inventory.
> blueprint-diff latest
from: blueprint 58d5e830-0884-47d8-a7cd-b2b3751adeb4
to:   blueprint af934083-59b5-4bf6-8966-6fb5292c29e1

 MODIFIED SLEDS:

  sled e96e226f-4ed9-4c01-91b9-69a9cd076c9e (active, config generation 2 -> 3):

    measurements:
    ---------------------------
    hash      version          
    ---------------------------
    unknown   (unknown version)


    host phase 2 contents:
    ------------------------
    slot   boot image source
    ------------------------
    A      current contents 
    B      current contents 


    physical disks:
    ------------------------------------------------------------------------------------
    vendor        model        serial                                        disposition
    ------------------------------------------------------------------------------------
    fake-vendor   fake-model   serial-1fa138c9-708a-4f08-b206-4ccf7436b9d9   in service 


    datasets:
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                            dataset id                             disposition   quota     reservation   compression
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/crucible                                                       0b9144d3-99d7-4675-9b48-92db5475fe20   in service    none      none          off        
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/crypt/zone                                                     75339bab-d7af-4180-8562-b1754e3b5e6d   in service    none      none          off        
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/crypt/zone/oxz_crucible_c1467d0e-b3de-4bd8-b36a-d8b36626badc   0fef4f3f-2eda-4342-a846-4c55a122c763   in service    none      none          off        
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/crypt/zone/oxz_ntp_c800ba17-240e-4b72-8ae6-afc30b6baa96        0355ac47-a47c-4a90-86ec-0c434eb8d376   in service    none      none          off        
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/crypt/debug                                                    e0d0222a-f585-4fc3-b04b-1c4e7efb1830   in service    100 GiB   none          gzip-9     
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/crypt/local_storage                                            459f5fdd-1fc1-43b0-b0d0-022298f8abda   in service    none      none          off        
    oxp_1fa138c9-708a-4f08-b206-4ccf7436b9d9/local_storage_unencrypted                                      5cf6fa86-965b-4d42-af5e-1973797bcb62   in service    none      none          off        


    omicron zones:
    ---------------------------------------------------------------------------------------------------------------------
    zone type      zone id                                image source                disposition   underlay IP          
    ---------------------------------------------------------------------------------------------------------------------
*   crucible       c1467d0e-b3de-4bd8-b36a-d8b36626badc   - install dataset           in service    fd00:1122:3344:107::4
     └─                                                   + artifact: version 1.0.0                                      
*   internal_ntp   c800ba17-240e-4b72-8ae6-afc30b6baa96   - install dataset           in service    fd00:1122:3344:107::3
     └─                                                   + artifact: version 1.0.0                                      


 COCKROACHDB SETTINGS:
    state fingerprint:::::::::::::::::   (none) (unchanged)
    cluster.preserve_downgrade_option:   (do not modify) (unchanged)

 METADATA:
    internal DNS version:::   1 (unchanged)
    external DNS version:::   1 (unchanged)
    target release min gen:   1 (unchanged)
    nexus gen::::::::::::::   1 (unchanged)

 OXIMETER SETTINGS:
    generation:   1 (unchanged)
    read from::   SingleNode (unchanged)


internal DNS:
  DNS zone: "control-plane.oxide.internal" (unchanged)
    unchanged names: 53 (records: 75)

external DNS:
  DNS zone: "oxide.example" (unchanged)
    unchanged names: 5 (records: 9)



