You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
stale environments in drop down for environments #72
The environments feature was recently contributed in #54. I use puppet with r10k which encourages the use of throw-away environments as a means of promoting fast feature development and continuous integration (by creating git branches).
As an example, I had a throw-away environment called agent which I was using to do some feature development for installing agents.
Once I was satisfied with the Git branch for agent, I merged it into production. After that, I deleted agent Git branch in my puppet control repository. At its next run, r10k automatically removed the agent environment.
What consideration should be given to "throw-away" environments?
I'm sorry for not getting back to you sooner, I had hoped that @plazonic, who implemented this support in #71 would comment.
I suspect the solution here is to have a cron-job prune obsolete environments, whether integrated into the core or as a new flag for the report-pruning.
Would that suit your needs do you think? (I appreciate there would still be a period of time where the orphaned branch/environment were visible, but I think that this is the simplest solution to implement.)
Oh, sorry, not looking at this that often, tagging works though...
I don't see how this could be done automatically - how is puppet summary supposed to know that an environment is obsolete? There is already an option for prune command that will remove obsoleted reports but which of those are in environments that can go away and which ones are relevant and should be kept?
I suppose it wouldn't be hard to add a prune option for choosing entire environments to remove but you'd then somehow to automate that (e.g. at the time you delete environment you'd have to also issue a prune command).
Now you say that I've looked at the database and you're right; I had thought there was a separate table, so we could run "DELETE FROM environments WHERE $used=0".
But as you suggest, if there is a report with an environment it'll stay until pruned.
I suspect this is a "wontfix" / "cantfix", or adding a manual "-prune-env=testing/x"
Probably better if it were a bit more generic, i.e. if --environment="foo" was given to then restrict all specified prune ops to just that environment. One could then combine that with days 0 to prune it all (I think, not sure what happens if days is set to 0).
The only issue that I have at the moment is that I have un-pruned reports in my puppet-summary reports directory, but no way to reach them, since the environment has been deleted.
Does that make sense? So, e.g.:
[root@gems-mgmt-syscfg-1 reports]# tree -L 1 /etc/puppetlabs/code/environments/
/etc/puppetlabs/code/environments/
└── production
1 directory, 0 files
So, there is only production environment, but in my puppet-summary reports directory, I have 4 reports for a single host with environment legato:
I also wrote a bit of code to check for "dangling" yaml report files - i.e. ones that are not referred to in the database. That should help here too. I should probably wait before I push that commit or it might get mixed up with the previous pull request?
Indeed, graph would be empty too if it wasn't finding these entries. And as this is quite long ago it would have to be on the orphaned tab (which I assume you clicked on and is empty as well)? Is this node listed on the main page (listing all environments), under orphaned nodes?
I see nothing obviously wrong in the code - any errors in puppet-summary logs? How about if you add to the URL "?accept=application/json" - should make it dump any nodes as json or null if it doesn't find any (so ..../environment/legato?accept=application/json). This would exclude page templating being at fault.
journalctl for puppet-summary service shows only HTTP access log style entries. Would I see errors there as well? Nothing in there looks like an error.
This particular node has a lot of reports, so it looks like I'm hitting a maximum of 50 for reports (see #74 ), however, I'm able to see the report from the orphaned environment by going to its specific URL. E.g.: http://localhost:3001/report/1958
OK, I see what the problem is - mea culpa - filter for environments really should have been part of the sql query. I also saw I was missing environment filter in sql queries used for producing graph stats (so numbers shown were way too big).
Fixed both in pending pull as well as added more pruning options.
PR merged, so I'm going to hope this is resolved now. Thanks to @corporate-gadfly for your patience, and @plazonic for being willing and able to resolve the problems :)
There is the pending issue of 50-reports shown by default, which I'lll look at shortly. I'm actually amazed nobody spotted that until now!
Activity
skx commentedon May 19, 2020
I'm sorry for not getting back to you sooner, I had hoped that @plazonic, who implemented this support in #71 would comment.
I suspect the solution here is to have a cron-job prune obsolete environments, whether integrated into the core or as a new flag for the report-pruning.
Would that suit your needs do you think? (I appreciate there would still be a period of time where the orphaned branch/environment were visible, but I think that this is the simplest solution to implement.)
plazonic commentedon May 19, 2020
Oh, sorry, not looking at this that often, tagging works though...
I don't see how this could be done automatically - how is puppet summary supposed to know that an environment is obsolete? There is already an option for prune command that will remove obsoleted reports but which of those are in environments that can go away and which ones are relevant and should be kept?
I suppose it wouldn't be hard to add a prune option for choosing entire environments to remove but you'd then somehow to automate that (e.g. at the time you delete environment you'd have to also issue a prune command).
skx commentedon May 19, 2020
Now you say that I've looked at the database and you're right; I had thought there was a separate table, so we could run "DELETE FROM environments WHERE $used=0".
But as you suggest, if there is a report with an environment it'll stay until pruned.
I suspect this is a "wontfix" / "cantfix", or adding a manual "-prune-env=testing/x"
corporate-gadfly commentedon May 19, 2020
@plazonic , @skx Thanks for commenting.
I could manually delete environments (if such a feature was available) and/or try to add a post-hook to
r10k.skx commentedon May 19, 2020
I could imagine:
Which would just run:
plazonic commentedon May 19, 2020
Probably better if it were a bit more generic, i.e. if --environment="foo" was given to then restrict all specified prune ops to just that environment. One could then combine that with days 0 to prune it all (I think, not sure what happens if days is set to 0).
corporate-gadfly commentedon May 19, 2020
I may be overthinking my "original ask".
The only issue that I have at the moment is that I have un-pruned reports in my puppet-summary
reportsdirectory, but no way to reach them, since the environment has been deleted.Does that make sense? So, e.g.:
So, there is only
productionenvironment, but in mypuppet-summaryreports directory, I have 4 reports for a single host with environmentlegato:Yet, when I go to

puppet-summaryto look atlegato, I don't see anything:Any way to see those orphaned reports?
I think eventually, they will drop off (after the 30 day prune)??
plazonic commentedon May 19, 2020
I don't know that they will drop off - if they are not shown in the interface then they are not in the db but it wouldn't hurt to verify. E.g.
sqlite3 /path/to/db/ps.db
select * from reports where environment='legato';
You haven't maybe ran prune with wrong reports dir or something similar?
plazonic commentedon May 19, 2020
I also wrote a bit of code to check for "dangling" yaml report files - i.e. ones that are not referred to in the database. That should help here too. I should probably wait before I push that commit or it might get mixed up with the previous pull request?
corporate-gadfly commentedon May 19, 2020
Sorry, I got sidetracked. The result of the SQL reported the identical 4 reports from the previous comment.
So, they are in the DB.
corporate-gadfly commentedon May 19, 2020
To summarize:
productionreportsdirectory referencing these orphaned environments.plazonic commentedon May 19, 2020
Indeed, graph would be empty too if it wasn't finding these entries. And as this is quite long ago it would have to be on the orphaned tab (which I assume you clicked on and is empty as well)? Is this node listed on the main page (listing all environments), under orphaned nodes?
I see nothing obviously wrong in the code - any errors in puppet-summary logs? How about if you add to the URL "?accept=application/json" - should make it dump any nodes as json or null if it doesn't find any (so ..../environment/legato?accept=application/json). This would exclude page templating being at fault.
corporate-gadfly commentedon May 19, 2020
journalctlforpuppet-summaryservice shows only HTTP access log style entries. Would I see errors there as well? Nothing in there looks like an error.corporate-gadfly commentedon May 19, 2020
This particular node has a lot of reports, so it looks like I'm hitting a maximum of 50 for reports (see #74 ), however, I'm able to see the report from the orphaned environment by going to its specific URL. E.g.: http://localhost:3001/report/1958
Fix handling of environment filtering - should be done in queries.
plazonic commentedon May 20, 2020
OK, I see what the problem is - mea culpa - filter for environments really should have been part of the sql query. I also saw I was missing environment filter in sql queries used for producing graph stats (so numbers shown were way too big).
Fixed both in pending pull as well as added more pruning options.
skx commentedon May 20, 2020
PR merged, so I'm going to hope this is resolved now. Thanks to @corporate-gadfly for your patience, and @plazonic for being willing and able to resolve the problems :)
There is the pending issue of 50-reports shown by default, which I'lll look at shortly. I'm actually amazed nobody spotted that until now!
corporate-gadfly commentedon May 20, 2020
Brilliant and confirmed. As expected, most nodes are showing up under

Orphanedtab.skx commentedon May 22, 2020
Available now: