Quantcast

Travis [extended tests] tag

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Travis [extended tests] tag

Andy Polyakov-2
In order to improve CI turn-around times Travis config in master branch
was tweaked to minimize the time it takes to process pull requests. This
is done by "short-circuiting" most expensive tests: sanitizers,
coverage, wine-based tests. Thing to keep in mind is that
"short-circuited" test come out as passed/green. Rationale is that if
minimum tests pass, the build should still be marked green on github.
Even though it gives somewhat deceiving picture, in sense that you get
green check mark for test that might have failed otherwise. Expensive
tests are marked with "EXTENDED_TEST=yes" on the build page, and one can
easily see if it was skipped by looking at time it took to skip it, it
should be ~1 minute.

At the same time it would be inappropriate to deny the mere possibility
to exercise complete test set even on per-pull-request basis. [Note that
complete tests are always executed for each repo-push.] For this reason
possibility to "opt-in" for expensive tests was arranged by adding
"[extended tests]" tag to *last* commit. If forgotten (in case you
reckoned that request is "worthy" extended tests), or claimed desired
afterwards, it's possible to simply amend the last commit, add the tag
and force push. In such case minimal tests would be effectively wasted
(because they will be executed twice), but overall it should still be
resource saving, since majority of pull requests won't require extended
testing.

And in the context it's worth keeping in mind that it's possible to skip
CI tests altogether by tagging commit with "[skip ci]". This option is
appropriate for commentary or documentation typo fixes, readme updates,
non-x86 code updates...
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Travis [extended tests] tag

Kurt Roeckx
On Sun, Feb 26, 2017 at 11:23:42PM +0100, Andy Polyakov wrote:

> In order to improve CI turn-around times Travis config in master branch
> was tweaked to minimize the time it takes to process pull requests. This
> is done by "short-circuiting" most expensive tests: sanitizers,
> coverage, wine-based tests. Thing to keep in mind is that
> "short-circuited" test come out as passed/green. Rationale is that if
> minimum tests pass, the build should still be marked green on github.
> Even though it gives somewhat deceiving picture, in sense that you get
> green check mark for test that might have failed otherwise. Expensive
> tests are marked with "EXTENDED_TEST=yes" on the build page, and one can
> easily see if it was skipped by looking at time it took to skip it, it
> should be ~1 minute.
>
> At the same time it would be inappropriate to deny the mere possibility
> to exercise complete test set even on per-pull-request basis. [Note that
> complete tests are always executed for each repo-push.] For this reason
> possibility to "opt-in" for expensive tests was arranged by adding
> "[extended tests]" tag to *last* commit. If forgotten (in case you
> reckoned that request is "worthy" extended tests), or claimed desired
> afterwards, it's possible to simply amend the last commit, add the tag
> and force push. In such case minimal tests would be effectively wasted
> (because they will be executed twice), but overall it should still be
> resource saving, since majority of pull requests won't require extended
> testing.
>
> And in the context it's worth keeping in mind that it's possible to skip
> CI tests altogether by tagging commit with "[skip ci]". This option is
> appropriate for commentary or documentation typo fixes, readme updates,
> non-x86 code updates...

Can you explain how to tag it?


Kurt

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Travis [extended tests] tag

Salz, Rich
> > tests was arranged by adding "[extended tests]" tag to *last* commit.
...
> > And in the context it's worth keeping in mind that it's possible to
> > skip CI tests altogether by tagging commit with "[skip ci]". This
...
>>
> Can you explain how to tag it?

Your commit message should have the literal text above in it.
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Travis [extended tests] tag

Andy Polyakov-2
In reply to this post by Kurt Roeckx
>> In order to improve CI turn-around times Travis config in master branch
>> was tweaked to minimize the time it takes to process pull requests. This
>> is done by "short-circuiting" most expensive tests: sanitizers,
>> coverage, wine-based tests. Thing to keep in mind is that
>> "short-circuited" test come out as passed/green. Rationale is that if
>> minimum tests pass, the build should still be marked green on github.
>> Even though it gives somewhat deceiving picture, in sense that you get
>> green check mark for test that might have failed otherwise. Expensive
>> tests are marked with "EXTENDED_TEST=yes" on the build page, and one can
>> easily see if it was skipped by looking at time it took to skip it, it
>> should be ~1 minute.
>>
>> At the same time it would be inappropriate to deny the mere possibility
>> to exercise complete test set even on per-pull-request basis. [Note that
>> complete tests are always executed for each repo-push.] For this reason
>> possibility to "opt-in" for expensive tests was arranged by adding
>> "[extended tests]" tag to *last* commit. If forgotten (in case you
>> reckoned that request is "worthy" extended tests), or claimed desired
>> afterwards, it's possible to simply amend the last commit, add the tag
>> and force push. In such case minimal tests would be effectively wasted
>> (because they will be executed twice), but overall it should still be
>> resource saving, since majority of pull requests won't require extended
>> testing.
>>
>> And in the context it's worth keeping in mind that it's possible to skip
>> CI tests altogether by tagging commit with "[skip ci]". This option is
>> appropriate for commentary or documentation typo fixes, readme updates,
>> non-x86 code updates...
>
> Can you explain how to tag it?

Oh! Just add it anywhere in commit message, or even subject line. There
is only one thing that is essential, it can't be broken between multiple
lines. In the nutshell [extented tests] (or [skip ci]) has to be
grep-able, that's all.

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Loading...