Skip to content

Autoupdater tries to commit file not present in project

Summary

After switching to defining classes using attr module class Project had field __files_to_update as raw dict ({}). It couses that such dictionary was shared by all instances of that class. This couses autoupdater to fail after trying to commit file from another project to diffrent project.

Steps to reproduce

Run autoupdater on some group of projects. If in one project autoupdater will make changes in some file which is not present in onother project it will try to commit it.

Expected behavior

Autoupdater should update all projects.

Actual behavior

Autoupdater fails on trying to commit file which is not present in project.

Relevant logs and/or screenshots

06:01:30 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:30,637 [INFO]: Updating project "repozytorium-danych-medycznych/services"
06:01:32 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:32,741 [INFO]: Reseting merge request "Resolve "Autoupdater update"" to updated source branch.
06:01:32 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:32,760 [INFO]: Pushing changes to origin.
06:01:36 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:36,344 [INFO]: Package "docker/python" is up-to-date: 3.7.4-stretch
06:01:36 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:36,541 [INFO]: Package "python/attrs" is up-to-date: 19.1.0
06:01:36 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:36,814 [INFO]: Package "python/cattrs" is up-to-date: 0.9.0
06:01:37 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:36,999 [INFO]: Package "python/emoji" is up-to-date: 0.5.3
06:01:37 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:37,267 [INFO]: Package "python/motor" is up-to-date: 2.0.0
06:01:37 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:37,686 [INFO]: Update available for package "python/ruamel.yaml": 0.16.0 -> 0.16.5
06:01:38 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:38,006 [INFO]: Update available for package "python/numpy": 1.17.0 -> 1.17.1
06:01:38 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:38,233 [INFO]: Update available for package "python/openstacksdk": 0.32.0 -> 0.35.0
06:01:38 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:38,575 [INFO]: Package "python/Pillow" is up-to-date: 6.1.0
06:01:38 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:38,777 [INFO]: Package "python/aiofiles" is up-to-date: 0.4.0
06:01:39 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:39,111 [INFO]: Package "python/aiohttp" is up-to-date: 3.5.4
06:01:39 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:39,385 [INFO]: Update available for package "python/sanic": 19.6.2 -> 19.6.3
06:01:39 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:39,770 [INFO]: Package "python/filetype" is up-to-date: 1.0.5
06:01:40 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:40,044 [INFO]: Package "python/progress" is up-to-date: 1.5
06:01:40 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:40,316 [INFO]: Package "python/pydicom" is up-to-date: 1.3.0
06:01:40 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:40,520 [INFO]: Package "python/requests" is up-to-date: 2.22.0
06:01:40 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:40,850 [INFO]: Update available for package "python/mypy": 0.711 -> 0.720
06:01:41 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:41,111 [INFO]: Sending build/base/requirements.txt commit
06:01:41 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:41,898 [INFO]: Sending build/image-service/requirements.txt commit
06:01:42 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:42,702 [INFO]: Sending build/service/requirements.txt commit
06:01:43 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:43,434 [INFO]: Sending build/build.sh commit
06:01:44 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:44,219 [INFO]: Sending CHANGELOG.md commit
06:01:45 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:45,065 [INFO]: End updating project "repozytorium-danych-medycznych/services"
06:01:45 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:45,515 [INFO]: Updating project "citask/monitoring/playbooks/services"
06:01:46 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:46,070 [INFO]: Package "python/ansible" is up-to-date: 2.8.4
06:01:46 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:46,257 [INFO]: Package "python/hvac" is up-to-date: 0.9.5
06:01:48 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:47,925 [INFO]: Package "dockerhub/influxdb" is up-to-date: 1.7.7
06:01:48 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:48,213 [INFO]: Package "ansible-roles/docker" is up-to-date: 4.0.1
06:01:48 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:48,580 [INFO]: Package "ansible-roles/grafana" is up-to-date: 1.1.17
06:01:48 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:48,876 [INFO]: Package "ansible-roles/iptables" is up-to-date: 1.1.0
06:01:49 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:49,164 [INFO]: Package "ansible-roles/kapacitor" is up-to-date: 1.2.0
06:01:49 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:49,498 [INFO]: Package "ansible-roles/lldp" is up-to-date: 1.1.0
06:01:49 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:49,691 [INFO]: Package "ansible-roles/networking" is up-to-date: 1.3.0
06:01:50 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:50,104 [INFO]: Package "ansible-roles/nginx" is up-to-date: 5.5.1
06:01:50 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:50,321 [INFO]: Package "ansible-roles/python" is up-to-date: 1.2.0
06:01:50 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:50,497 [INFO]: Package "ansible-roles/service" is up-to-date: 1.5.2
06:01:50 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:50,709 [INFO]: Package "ansible-roles/ssh" is up-to-date: 3.2.0
06:01:50 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:50,891 [INFO]: Package "ansible-roles/system" is up-to-date: 1.4.0
06:01:51 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:51,529 [INFO]: Package "ansible-roles/telegraf" is up-to-date: 2.4.1
06:01:51 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:51,529 [INFO]: Creating new issue in "services" project
06:01:51 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:51,774 [INFO]: Creating new branch in "services" project
06:01:52 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:52,681 [INFO]: Creating new merge request in "Autoupdater update" issue in "services" project
06:01:53 [rundeck@0.0.0.0 1][NORMAL] 2019-08-29 06:01:53,140 [INFO]: Sending build/base/requirements.txt commit
06:01:53 [rundeck@0.0.0.0 1][NORMAL] Traceback (most recent call last):
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/gitlab/exceptions.py", line 259, in wrapped_f
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     return f(*args, **kwargs)
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/gitlab/mixins.py", line 204, in create
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     server_data = self.gitlab.http_post(path, post_data=data, files=files, **kwargs)
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/gitlab/__init__.py", line 669, in http_post
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     **kwargs
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/gitlab/__init__.py", line 564, in http_request
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     response_body=result.content,
06:01:53 [rundeck@0.0.0.0 1][NORMAL] gitlab.exceptions.GitlabHttpError: 400: A file with this name doesn't exist
06:01:53 [rundeck@0.0.0.0 1][NORMAL] 
06:01:53 [rundeck@0.0.0.0 1][NORMAL] During handling of the above exception, another exception occurred:
06:01:53 [rundeck@0.0.0.0 1][NORMAL] 
06:01:53 [rundeck@0.0.0.0 1][NORMAL] Traceback (most recent call last):
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/bin/autoupdater", line 10, in <module>
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     sys.exit(main())
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/autoupdater/commander.py", line 34, in main
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     arguments.webhook
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/autoupdater/projects/update.py", line 14, in update_projects
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     updated_projects = list(repository.update_projects(project_to_update, labels))
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/autoupdater/gitlab/update.py", line 90, in update_projects
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     updated_project = project.update(labels)
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/autoupdater/gitlab/update.py", line 203, in update
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     self.__commit_changes(merge_request)
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/autoupdater/gitlab/update.py", line 269, in __commit_changes
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     self.__make_commit(merge_request, file_to_update, self.__files_to_update[file_to_update])
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/autoupdater/gitlab/update.py", line 265, in __make_commit
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     self.__project.commits.create(data)
06:01:53 [rundeck@0.0.0.0 1][NORMAL]   File "/usr/local/lib/python3.7/site-packages/gitlab/exceptions.py", line 261, in wrapped_f
06:01:53 [rundeck@0.0.0.0 1][NORMAL]     raise error(e.error_message, e.response_code, e.response_body)
06:01:53 [rundeck@0.0.0.0 1][NORMAL] gitlab.exceptions.GitlabCreateError: 400: A file with this name doesn't exist
06:01:54 [rundeck@0.0.0.0 1][NORMAL] Running cleanup - removing workspace /var/lib/rundeck/work/teamwork/711000
06:01:54 [rundeck@0.0.0.0 1][ERROR] Result: 1
06:01:54 [rundeck@0.0.0.0 1][ERROR] Failed: NonZeroResultCode: Result code was 1

Possible fixes

Change definition of field __files_to_update in class Project from __files_to_update: typing.Dict[str, str] = {} to __files_to_update: typing.Dict[str, str] = attr.ib(factory=dict)