Project

General

Profile

Actions
Copy link

Bug #37332

closed

Katello::Errors::Pulp3Error: module 'createrepo_c' has no attribute 'SHA1'

Added by Michael Arlt 25 days ago. Updated 8 days ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
RPMs
Target version:
Foreman - 3.11.0
Difficulty:
Triaged:
No
Bugzilla link:
Pull request:
https://github.com/theforeman/pulpcore-packaging/pull/969
Fixed in Releases:
Found in Releases:
Katello - Katello 4.12.0, Katello - Katello 4.13.0
Red Hat JIRA:

Description

Tasks Synchronize repository

Tab Task shows Result: warning

Tab Errors shows:

Action:

Actions::Pulp3::Repository::CreatePublication

Input:
{"repository_id"=>70,
 "smart_proxy_id"=>1,
 "contents_changed"=>true,
 "skip_publication_creation"=>false,
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
 "current_timezone"=>"Europe/Berlin",
 "current_organization_id"=>1,
 "current_location_id"=>2,
 "current_user_id"=>5}

Output:

{"response"=>
  {"task"=>"/pulp/api/v3/tasks/018ec330-5864-768d-92d6-0e23722b194f/"},
 "pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/018ec330-5864-768d-92d6-0e23722b194f/",
    "pulp_created"=>"2024-04-09T14:08:29.286+00:00",
    "state"=>"failed",
    "name"=>"pulp_rpm.app.tasks.publishing.publish",
    "logging_cid"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
    "created_by"=>"/pulp/api/v3/users/1/",
    "started_at"=>"2024-04-09T14:09:53.181+00:00",
    "finished_at"=>"2024-04-09T14:09:55.293+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/lib/python3.11/site-packages/pulpcore/tasking/tasks.py\", line 61, in _execute_task\n" +
       "    result = func(*args, **kwargs)\n" +
       "             ^^^^^^^^^^^^^^^^^^^^^\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 384, in publish\n" +
       "    generate_repo_metadata(\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 686, in generate_repo_metadata\n" +
       "    checksum_type = cr_checksum_type_from_string(\n" +
       "                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n" +
       "  File \"/usr/lib/python3.11/site-packages/pulp_rpm/app/tasks/publishing.py\", line 318, in cr_checksum_type_from_string\n" +
       "    return getattr(cr, checksum_type.upper())\n" +
       "           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "description"=>"module 'createrepo_c' has no attribute 'SHA1'"},
    "worker"=>"/pulp/api/v3/workers/018ec2e7-e086-7e2f-9ff7-7e7f057ffa05/",
    "child_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Generating repository metadata",
       "code"=>"publish.generating_metadata",
       "state"=>"failed",
       "total"=>1,
       "done"=>0}],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["shared:/pulp/api/v3/repositories/rpm/rpm/018ec303-a1fd-7ff7-af7b-81104bf6aecb/",
      "shared:/pulp/api/v3/domains/018ec258-d879-75ba-a85e-e6f71108fc5c/"]}],
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>14, "failed"=>1}}

Exception:

Katello::Errors::Pulp3Error: module 'createrepo_c' has no attribute 'SHA1'

Backtrace:

/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:108:in `block in check_for_errors'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:106:in `each'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:106:in `check_for_errors'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:162:in `poll_external_task'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/polling.rb:22:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/cancellable.rb:14:in `run'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/pulp3/abstract_async_task.rb:10:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:589:in `block (3 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/execute_if_contents_changed.rb:5:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/usr/share/gems/gems/katello-4.12.0/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/usr/share/gems/gems/katello-4.12.0/app/lib/actions/middleware/remote_action.rb:16:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/execution_wrapper.rb:91:in `wrap'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action/progress.rb:17:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
/usr/share/gems/gems/foreman-tasks-9.1.1/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/middleware/world.rb:31:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:588:in `block (2 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:587:in `catch'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:587:in `block in execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `block in with_error_handling'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `catch'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:490:in `with_error_handling'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:582:in `execute_run'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/action.rb:303:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/director.rb:94:in `execute'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors.rb:18:in `run_user_code'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:202:in `execute_job'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/middleware/chain.rb:172:in `invoke'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:169:in `block in process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:113:in `local'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq.rb:44:in `block in <module:Sidekiq>'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:263:in `stats'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:13:in `call'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_retry.rb:80:in `global'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:124:in `block in dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/job_logger.rb:39:in `prepare'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:123:in `dispatch'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:168:in `process'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:78:in `process_one'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/processor.rb:68:in `run'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:8:in `watchdog'
/usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/component.rb:17:in `block in safe_thread'
/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Action:

Actions::Pulp3::Repository::SavePublication

Input:

{"repository_id"=>70,
 "tasks"=>Step(12).output,
 "contents_changed"=>Step(9).output[:contents_changed],
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>"be69d830-823f-47fc-84e6-e9a815aed3a3",
 "current_timezone"=>"Europe/Berlin",
 "current_organization_id"=>1,
 "current_location_id"=>2,
 "current_user_id"=>5}

Output:

{}

Actions
Copy link
 #2

Updated by Michael Arlt 25 days ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by Michael Arlt 25 days ago

I am not 100% sure but i think that the synchronisation worked with 3.10.0 rc1 or rc2.

Actions
Copy link
 #4

Updated by Michael Arlt 24 days ago

Adding pulpcore::allowed_content_checksums: ['sha1', 'sha224', 'sha256', 'sha384', 'sha512']
to custom-hiera.yaml creates

ALLOWED_CONTENT_CHECKSUMS = ["sha1", "sha224", "sha256", "sha384", "sha512"]

in /etc/pulp/settings.py but the problem persists.

Actions
Copy link
 #5

Updated by Samir Jha 24 days ago

  • Status changed from New to Need more information
  • Assignee set to Samir Jha
  • Triaged changed from No to Yes

Hi Michael,

Could you open an issue with pulp team here: https://github.com/pulp/pulp_rpm/issues.

We can keep this issue open if this ends up being a katello issue or a configuration issue.

Actions
Copy link
 #7

Updated by Daniel Alley 23 days ago

I believe this is a build issue with the "legacy_hashes" flag not being enabled for the RHEL 9 build of the package https://github.com/rpm-software-management/createrepo_c/blob/master/createrepo_c.spec#L27-L31

I don't believe we've modified those lines from upstream and thus it would likely not be enabled by default. So we should tweak those lines or set that flag.

Actions
Copy link
 #9

Updated by Michael Arlt 18 days ago

@Samir: Could you please let me know if and when there will be a solution to the problem?
I really don't want to cause any stress - this is just for my planning.

Actions
Copy link
 #10

Updated by Michael Arlt 15 days ago

  • Found in Releases Katello 4.13.0, Katello Next added

reproducible with Katello 4.13 nightly:

$ rpm -qa|grep -i katello
katello-repos-4.13-0.1.nightly.el9.noarch
katello-common-4.13.0-0.1.master.el9.noarch
katello-certs-tools-2.10.0-1.el9.noarch
katello-selinux-5.0.2-1.el9.noarch
foreman-installer-katello-3.11.0-0.1.develop.20240418031237gitd83807b.el9.noarch
rubygem-katello-4.13.0-0.1.pre.master.20240411211935git58d2cdb.el9.noarch
katello-4.13.0-0.1.master.el9.noarch
rubygem-hammer_cli_katello-1.12.0-0.1.pre.master.el9.noarch
katello-client-bootstrap-1.7.9-2.el9.noarch

Actions
Copy link
 #11

Updated by Michael Arlt 11 days ago

  • Found in Releases deleted (Katello Next)
Actions
Copy link
 #12

Updated by Michael Arlt 11 days ago

  • Triaged changed from Yes to No
Actions
Copy link
 #13

Updated by Michael Arlt 11 days ago

  • Status changed from Need more information to Feedback
Actions
Copy link
 #14

Updated by Michael Arlt 11 days ago

  • Status changed from Feedback to New
Actions
Copy link
 #15

Updated by Samir Jha 11 days ago

Hi Michael,

Looking at Daniel's comment on the pulp issue, this doesn't seem like something we can fix in katello. I'd propose closing this issue and following up on the community?

Actions
Copy link
 #16

Updated by Michael Arlt 11 days ago

Hi Samir,

I don't know where we should open the issue but it is not possible to use SHA1 Repos in Foreman 3.10 on RHEL 9.
Please don't close - instead please ask others in the foreman project where it should addressed.

Actions
Copy link
 #17

Updated by Ian Ballou 9 days ago

  • Status changed from New to Need more information

This issue cannot be fixed by Pulp or Foreman directly if the cause is due to a built version of createrepo_c that doesn't allow dealing with SHA1 content. I'll leave the issue open for now so we don't forget about it, but the fix will require finding who is in charge of building createrepo_c and seeing if enabling SHA1 via the flag that Daniel mentioned is possible.

Edit: as it turns out, we are packaging createrepo_c, I thought it was coming from the EL9 repos. Changing this into a foreman-packaging issue. To support syncing SHA1 content, we will need to enable that legacy flag in our build of createrepo_c.

Actions
Copy link
 #18

Updated by Ian Ballou 9 days ago

  • Project changed from Katello to Packaging
  • Category changed from Sync Plans to RPMs
  • Assignee deleted (Samir Jha)
  • Target version set to 3.11.0
Actions
Copy link
 #19

Updated by Ian Ballou 9 days ago

  • Pull request https://github.com/theforeman/pulpcore-packaging/pull/969 added
Actions
Copy link
 #20

Updated by Michael Arlt 9 days ago

Perfect - thank you!

Any chance to see this fix in a 3.10 build?

Actions
Copy link
 #21

Updated by Odilon Sousa 9 days ago

Michael Arlt wrote in #note-20:

Perfect - thank you!

Any chance to see this fix in a 3.10 build?

It's published now, you can find the pulpcore 3.39 repo.

Actions
Copy link
 #22

Updated by Michael Arlt 8 days ago

It's published now, you can find the pulpcore 3.39 repo.

Works! Thank you so much!

Actions
Copy link
 #23

Updated by Ian Ballou 8 days ago

  • Status changed from Need more information to Closed

Closing this out now since it sounds like it's all taken care of.

Actions
Copy link

Also available in: Atom PDF