how I ride the bullet - how I set up a testing environment

Discussions and requests related to new CMSimple features, plugins, templates etc. and how to develop.
Please don't ask for support at this forums!
cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Tue Dec 30, 2014 12:04 pm

manu wrote:Christoph, do you know what this is?
Ah, yes, that test would need to be rewritten, but the necessary CURLFile class is only available since PHP 5.5.0. The cleanest solution would be duplicating the test logic (if (class_exists(...), or something), but it might be acceptable to just silence the test with an @ operator. I'll look into that.
manu wrote:phing coverage doesn't write anything.
That might be due to unsufficient permissions to create the folder tests/coverage. Please create this folder manually, and try again. If that works, it might be best to add the (empty) folder to the repo and ignore its contents only.

PS:
manu wrote:phing all-tests seem to stop after unit-tests
That doesn't happen to me. :?
Christoph M. Becker – Plugins for CMSimple_XH

manu
Posts: 819
Joined: Wed Jun 04, 2008 12:05 pm
Location: St. Gallen - Schweiz
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by manu » Tue Dec 30, 2014 1:59 pm

cmb wrote:Please create this folder manually, and try again.
same result:

Code: Select all

C:\xampp\htdocs\CMSimpleDev>phing coverage -logger phing.listener.DefaultLogger
Buildfile: C:\xampp\htdocs\CMSimpleDev\build.xml

CMSimple_XH > coverage:

PHPUnit 4.4.0 by Sebastian Bergmann.

Configuration read from C:\xampp\htdocs\CMSimpleDev\coverage.xml

...............................................................  63 / 560 ( 11%)

............................................................... 126 / 560 ( 22%)

.............
BUILD FINISHED

Total time: 21.3330 seconds


C:\xampp\htdocs\CMSimpleDev>
tests/coverage/ remains empty.
cmb wrote:That doesn't happen to me. :?
And this is the log of phing all-tests:

Code: Select all

C:\xampp\htdocs\CMSimpleDev>phing all-tests -logger phing.listener.DefaultLogger

Buildfile: C:\xampp\htdocs\CMSimpleDev\build.xml

CMSimple_XH > unit-tests:

  [phpunit] Testsuite: AdminfuncsTest
  [phpunit] Tests run: 3, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.02000 s
  [phpunit] Testsuite: AdminMenuTest::testShowsItem
  [phpunit] Tests run: 19, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.57000 s
  [phpunit] Testsuite: AdminMenuTest::testShowsPluginsInColumns
  [phpunit] Tests run: 4, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.13000 s
  [phpunit] Testsuite: AdminMenuTest
  [phpunit] Tests run: 29, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.85000 s
  [phpunit] Testsuite: BackupTest::testIsContentBackup
  [phpunit] Tests run: 10, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.15000 s

C:\xampp\htdocs\CMSimpleDev>

cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Tue Dec 30, 2014 2:20 pm

manu wrote:
cmb wrote:Please create this folder manually, and try again.
same result:
[ console output ]
tests/coverage/ remains empty.
Do you have Xdebug installed?
manu wrote:
cmb wrote:That doesn't happen to me. :?
And this is the log of phing all-tests:

Code: Select all

[ ... ]
  [phpunit] Testsuite: BackupTest::testIsContentBackup
  [phpunit] Tests run: 10, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.15000 s

C:\xampp\htdocs\CMSimpleDev>
Ah! Apparently the unit tests fail early, probably during BackupTest::testFailedBackupReportsError() where I had to turn down error reporting temporarily (can't remember why; should have documented the hack :oops:). I'll investigate.

PS: During investigating the issues I found a new one: the last test of SearchTest::testSearch() fails, if the intl extension is disabled. <del>I'll fix this test ASAP.</del><ins>I've fixed this test with r1426.</ins>

Furthermore I've noticed that there are new versions of XAMPP (I'm still using 1.8.2 and occassionally 1.8.3). Which version do you use, manu?

PPS: When searching for "&" in edit mode three results are found; amongst them "News02" which I find pretty strange. Not a real problem, but something that should be investigated.
Christoph M. Becker – Plugins for CMSimple_XH

manu
Posts: 819
Joined: Wed Jun 04, 2008 12:05 pm
Location: St. Gallen - Schweiz
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by manu » Tue Dec 30, 2014 4:59 pm

My XAMPP Version number is 5.5.19 (same as PHP Version).
Revision 1426? You probably have to push this onto the server. In my log the latest revision is 1425.

cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Tue Dec 30, 2014 5:12 pm

manu wrote:My XAMPP Version number is 5.5.19 (same as PHP Version).
Thanks. Anyhow, in the meantime I have been able to reproduce the silently failing test suite (including the failing coverage report) with XAMPP 1.8.3. I'll see to it this evening.
manu wrote:Revision 1426? You probably have to push this onto the server. In my log the latest revision is 1425.
It is there for roughly an hour. You may have to update your working copy.
Christoph M. Becker – Plugins for CMSimple_XH

cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Tue Dec 30, 2014 6:09 pm

cmb wrote:Ah! Apparently the unit tests fail early, probably during BackupTest::testFailedBackupReportsError() where I had to turn down error reporting temporarily (can't remember why; should have documented the hack :oops:). I'll investigate.
The reason for temporarily disabling error_reporting is that the test checks for a failing backup, and we can't use a try-catch statement in this case (otherwise e() won't be called, what is checked by the test spy). I've added a respective comment (r1427). Alternatively, we could use the @ operator here, what is simpler, but not as explicit. When I've written the code back then, I tried to avoid @ at all cost, because I generally consider it bad practise, but for unit-tests we might have to live with it (actually, disabling the error_reporting explicitly is not better).[1]

Anyhow, just during `$this->_subject->execute();` we need a function nesting level of more than 100 (when the tests are run by Phing), but that is prohibited by Xdebug's default settings. The solution is to add the following to php.ini:

Code: Select all

xdebug.max_nesting_level = 200
I have documented this in the Wiki article just now. (Probably 110 would be sufficient, but better safe than sorry -- and I don't expect any problems with this rather high value.)

PS: [1] The cleanest solution might be to use exception handling in the core, but that's not an option for now (PHP 4 support).
Christoph M. Becker – Plugins for CMSimple_XH

manu
Posts: 819
Joined: Wed Jun 04, 2008 12:05 pm
Location: St. Gallen - Schweiz
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by manu » Wed Dec 31, 2014 12:28 pm

aah, all targets run now as expected, even the coverage target, I suppose because of the max_nesting_level. To get the CSSFAttack run I added CURL_SAFE_UPLOAD option (see Revision Log 1428). Christoph please check if this is correct.

cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Wed Dec 31, 2014 1:45 pm

manu wrote:aah, all targets run now as expected, even the coverage target, I suppose because of the max_nesting_level.
Yes (at least that was so for me). Thanks for the confirmation.
manu wrote:To get the CSSFAttack run I added CURL_SAFE_UPLOAD option (see Revision Log 1428). Christoph please check if this is correct.
Great! I've already forgotten that issue. Using CURL_SAFE_UPLOAD is a very good idea, but I suppose it will fail for PHP 5.6, where the meaning of the option has been reversed :roll:, according to the RFC. The manual page of curl_setopt() doesn't list the new option. I'll try to verify the issue, but unfortunately, there is no runkit.dll available for PHP 5.6. :cry:
Christoph M. Becker – Plugins for CMSimple_XH

cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Fri Jan 02, 2015 2:39 pm

cmb wrote:The manual page of curl_setopt() doesn't list the new option.
That has already been reported a while ago: https://bugs.php.net/bug.php?id=66677.
cmb wrote:there is no runkit.dll available for PHP 5.6.
The zenovich fork doesn't even build on PHP 5.6, but the adrianguenter fork does. :)
cmb wrote:Using CURL_SAFE_UPLOAD is a very good idea, but I suppose it will fail for PHP 5.6, where the meaning of the option has been reversed :roll:, according to the RFC.
I have misread the description in the RFC, so r1428 is fine. :)
Christoph M. Becker – Plugins for CMSimple_XH

cmb
Posts: 13313
Joined: Tue Jun 21, 2011 11:04 am
Location: Mü-Sa, RLP, DE
Contact:

Re: how I ride the bullet - how I set up a testing environme

Post by cmb » Thu Jan 08, 2015 9:04 pm

Well, I just run `phing attack-tests` under PHP 5.4.19[1] and that failed because CURLOPT_SAFE_UPLOAD has been introduced only in PHP 5.5.0. I wanted to fix that with a conditionally set option, but I've noticed that setting the option to TRUE doesn't work as intended -- instead of uploading hack.txt, this simply sends a parameter with the filename (prepended by @). That makes the test bogus.

The option would have to be set to FALSE (i.e. allow unsafe file uploads), but that triggers a deprecation warning. So the clean solution would be to use the CURLFile class for PHP >= 5.5.0. For now (r1440) I've added an @ operator, and a respective @todo. I recommend to fix that (and other @ operators in the tests) ASAP.

[1] I suggest to keep the tests and other supporting development scripts compatible to all supported PHP versions, at least.
Christoph M. Becker – Plugins for CMSimple_XH

Post Reply