i have teamcity build captures code coverage unit tests. have defined environment variable minimum code coverage build succeed, works fine don't maintain threshold manually. question have whether there way (outside of publishing code coverage statistics somewhere outside of teamcity , reading results last successful build) automatically adjust threshold code coverage improves ensure it's steady improvement without allowing backsliding :)?
for example, suppose current code coverage 20% (a legacy application) , new unit tests written, code coverage improves 25%. then, checks in new code without unit tests , code coverage drops 24%. want teamcity fail build because code coverage dropped 25% 24%.
i have pet theories code coverage i'd explain first before on answering question.
first context:
- there many sorts of code coverage, i'm going talk line coverage, should able substitute different sort.
- from question: "...someone checks in new code without unit tests , code coverage drops..." related similar: "some 1 (refactors/eliminates duplication/replaces algorithm and) removes tested code , coverage drops."
- coverage should measured result of running single suite of tests. is, not running application , stimulating outside.
- coverage percentage misleading.
had think , want know how many lines of code not covered.
see comment answer: ensure minimal coverage on new subversion commits - coverage should high possible. question talks "...improvement without allowing backsliding..."
- 100% coverage possible.
i've done it, albeit library.
i have theory should divide code 2 divisions far code coverage concerned:
- a division code 100% covered.
- a division no code covered.
either division constituted number of projects, members of division should files (given both java , c# have source files) , preferably whole folders of files. have 1 set of project in first division , set in second division.
now report of lack of coverage number of lines in second division.
the mode of operation should testing code go , code falls 100% coverage division. however, if find tricky piece of code brain can't find way test, should refactor bits aren't tested move second division. alternatively may brainwave , able find test raises second division above 0%, @ point refactor code on first division. means every check-in maintains theoretical invariant.
now, question:
no, don't know teamcity @ apart brief @ jetbrains website, don't know how update coverage, according theory should 100% or nothing, can set limits per project? if can, fixed limit of 100% works first division.
if can 2 divisions may want automatic update thing lines of code metric second division, progressively lower better.
Comments
Post a Comment