Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

stale environments in drop down for environments #72

Closed

Description

@corporate-gadfly

The environments feature was recently contributed in #54. I use puppet with r10k which encourages the use of throw-away environments as a means of promoting fast feature development and continuous integration (by creating git branches).

As an example, I had a throw-away environment called agent which I was using to do some feature development for installing agents.
image

Once I was satisfied with the Git branch for agent, I merged it into production. After that, I deleted agent Git branch in my puppet control repository. At its next run, r10k automatically removed the agent environment.

What consideration should be given to "throw-away" environments?

Activity

  1. skx commented on May 19, 2020

    @skx
    Owner

    I'm sorry for not getting back to you sooner, I had hoped that @plazonic, who implemented this support in #71 would comment.

    I suspect the solution here is to have a cron-job prune obsolete environments, whether integrated into the core or as a new flag for the report-pruning.

    Would that suit your needs do you think? (I appreciate there would still be a period of time where the orphaned branch/environment were visible, but I think that this is the simplest solution to implement.)

  2. self-assigned this
    on May 19, 2020
  3. plazonic commented on May 19, 2020

    @plazonic

    Oh, sorry, not looking at this that often, tagging works though...

    I don't see how this could be done automatically - how is puppet summary supposed to know that an environment is obsolete? There is already an option for prune command that will remove obsoleted reports but which of those are in environments that can go away and which ones are relevant and should be kept?

    I suppose it wouldn't be hard to add a prune option for choosing entire environments to remove but you'd then somehow to automate that (e.g. at the time you delete environment you'd have to also issue a prune command).

  4. skx commented on May 19, 2020

    @skx
    Owner

    Now you say that I've looked at the database and you're right; I had thought there was a separate table, so we could run "DELETE FROM environments WHERE $used=0".

    But as you suggest, if there is a report with an environment it'll stay until pruned.

    I suspect this is a "wontfix" / "cantfix", or adding a manual "-prune-env=testing/x"

  5. corporate-gadfly commented on May 19, 2020

    @corporate-gadfly
    ContributorAuthor

    @plazonic , @skx Thanks for commenting.

    I could manually delete environments (if such a feature was available) and/or try to add a post-hook to r10k.

  6. skx commented on May 19, 2020

    @skx
    Owner

    I could imagine:

    $ ./puppet-summary  prune -environment="foo"
    

    Which would just run:

    SQL>  UPDATE reports SET environment='' WHERE environment='foo'
    
  7. plazonic commented on May 19, 2020

    @plazonic

    Probably better if it were a bit more generic, i.e. if --environment="foo" was given to then restrict all specified prune ops to just that environment. One could then combine that with days 0 to prune it all (I think, not sure what happens if days is set to 0).

  8. corporate-gadfly commented on May 19, 2020

    @corporate-gadfly
    ContributorAuthor

    I may be overthinking my "original ask".

    The only issue that I have at the moment is that I have un-pruned reports in my puppet-summary reports directory, but no way to reach them, since the environment has been deleted.

    Does that make sense? So, e.g.:

    [root@gems-mgmt-syscfg-1 reports]# tree -L 1 /etc/puppetlabs/code/environments/
    /etc/puppetlabs/code/environments/
    └── production
    
    1 directory, 0 files
    

    So, there is only production environment, but in my puppet-summary reports directory, I have 4 reports for a single host with environment legato:

    [root@gems-mgmt-syscfg-1 reports]# grep -rl 'environment: legato' .
    ./easi-sh2-uat-rxpws-1.fqdn/96071da26932820e87cf6e12c8c2b9ea4d2f43c4
    ./easi-sh2-uat-rxpws-1.fqdn/aa416bcb2d972cc0fd9b8762d8f1a62bca1a1cf9
    ./easi-sh2-uat-rxpws-1.fqdn/e76e01ddf350006e466d754e9c19cbf161389213
    ./easi-sh2-uat-rxpws-1.fqdn/0b2149f5d0f02cb8897fb52f2e085308cef4e4cb
    

    Yet, when I go to puppet-summary to look at legato, I don't see anything:
    image
    Any way to see those orphaned reports?

    I think eventually, they will drop off (after the 30 day prune)??

  9. plazonic commented on May 19, 2020

    @plazonic

    I don't know that they will drop off - if they are not shown in the interface then they are not in the db but it wouldn't hurt to verify. E.g.

    sqlite3 /path/to/db/ps.db
    select * from reports where environment='legato';

    You haven't maybe ran prune with wrong reports dir or something similar?

  10. plazonic commented on May 19, 2020

    @plazonic

    I also wrote a bit of code to check for "dangling" yaml report files - i.e. ones that are not referred to in the database. That should help here too. I should probably wait before I push that commit or it might get mixed up with the previous pull request?

  11. corporate-gadfly commented on May 19, 2020

    @corporate-gadfly
    ContributorAuthor

    sqlite3 /path/to/db/ps.db
    select * from reports where environment='legato';

    Sorry, I got sidetracked. The result of the SQL reported the identical 4 reports from the previous comment.

    1958|easi-sh2-uat-rxpws-1|failed|easi-sh2-uat-rxpws-1./0b2149f5d0f02cb8897fb52f2e085308cef4e4cb|13.590671818|1588094655|272|3|1|0|legato
    1959|easi-sh2-uat-rxpws-1|changed|easi-sh2-uat-rxpws-1./96071da26932820e87cf6e12c8c2b9ea4d2f43c4|18.73729137|1588094729|272|0|0|1|legato
    1961|easi-sh2-uat-rxpws-1|unchanged|easi-sh2-uat-rxpws-1/aa416bcb2d972cc0fd9b8762d8f1a62bca1a1cf9|16.857331314|1588094883|272|0|0|0|legato
    1962|easi-sh2-uat-rxpws-1|changed|easi-sh2-uat-rxpws-1/e76e01ddf350006e466d754e9c19cbf161389213|41.500788158|1588095283|272|0|0|2|legato
    

    I don't know that they will drop off - if they are not shown in the interface then they are not in the db but it wouldn't hurt to verify.

    So, they are in the DB.

  12. corporate-gadfly commented on May 19, 2020

    @corporate-gadfly
    ContributorAuthor

    To summarize:

    1. There is only one real environment, production
    2. There are orphaned environments which are shown in the dropdown.
    • There are files present in the reports directory referencing these orphaned environments.
    • There are entries present in the database referencing these orphaned environments
    1. When selecting the orphaned environment from the dropdown, no reports are shown
  13. plazonic commented on May 19, 2020

    @plazonic

    Indeed, graph would be empty too if it wasn't finding these entries. And as this is quite long ago it would have to be on the orphaned tab (which I assume you clicked on and is empty as well)? Is this node listed on the main page (listing all environments), under orphaned nodes?

    I see nothing obviously wrong in the code - any errors in puppet-summary logs? How about if you add to the URL "?accept=application/json" - should make it dump any nodes as json or null if it doesn't find any (so ..../environment/legato?accept=application/json). This would exclude page templating being at fault.

  14. corporate-gadfly commented on May 19, 2020

    @corporate-gadfly
    ContributorAuthor
    • All tabs are empty when looking at orphaned environment
    • That particular node does appear under all environments and has timings and report entries
      image
    # curl -i -H "Accept: application/json" http://localhost:3001/environment/legato
    HTTP/1.1 200 OK
    Content-Type: application/json
    Date: Tue, 19 May 2020 22:05:40 GMT
    Content-Length: 4
    
    null
    

    journalctl for puppet-summary service shows only HTTP access log style entries. Would I see errors there as well? Nothing in there looks like an error.

  15. corporate-gadfly commented on May 19, 2020

    @corporate-gadfly
    ContributorAuthor

    This particular node has a lot of reports, so it looks like I'm hitting a maximum of 50 for reports (see #74 ), however, I'm able to see the report from the orphaned environment by going to its specific URL. E.g.: http://localhost:3001/report/1958

  16. added a commit that references this issue on May 20, 2020
    c9d7eb9
  17. plazonic commented on May 20, 2020

    @plazonic

    OK, I see what the problem is - mea culpa - filter for environments really should have been part of the sql query. I also saw I was missing environment filter in sql queries used for producing graph stats (so numbers shown were way too big).

    Fixed both in pending pull as well as added more pruning options.

  18. skx commented on May 20, 2020

    @skx
    Owner

    PR merged, so I'm going to hope this is resolved now. Thanks to @corporate-gadfly for your patience, and @plazonic for being willing and able to resolve the problems :)

    There is the pending issue of 50-reports shown by default, which I'lll look at shortly. I'm actually amazed nobody spotted that until now!

  19. corporate-gadfly commented on May 20, 2020

    @corporate-gadfly
    ContributorAuthor

    Brilliant and confirmed. As expected, most nodes are showing up under Orphaned tab.
    image

  20. skx commented on May 22, 2020

    @skx
    Owner
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    ⌂ 换网址