Use getHeader('unencoded_url') for IIS instead of $this->getHeader('request-uri')

[jatubio]

Hi simon,

I'm using IIS and i'm not sure if this issue happens also on Apache and another Web Servers.

When i have one url with utf8 characters, like: http://www.misite.com/especialidad/Dermatolog%C3%ADa or http://www.misite.com/especialidad/Dermatología, i'm getting a strangely encoded version of Dermatología, but not Dermatología or Dermatolog%C3%ADa

Strangely, for something like http://www.misite.com/?especialidad=Dermatolog%C3%ADa or http://www.misite.com/?especialidad=Dermatología i get Dermatología or Dermatolog%C3%ADa for $_GET['especialidad']

I've researched and, apparently, IIS encodes somehow the part that corresponds to the path of the url, but not on querystring part.

You can get more information here: tjanczuk/iisnode#486 and here: https://forums.iis.net/t/1153679.aspx

Then, i have made this change on __construct() method of Peece\Http\Request class:
$this->uri = urldecode($this->getHeader('unencoded_url')) ?: $this->getHeader('request-uri');

And it's working like a charm.

unencoded_url seems to be present only when url contains utf8 characters.

[skipperbent]

Thanks for mentioning this - haven't done a lot of testing on IIS to be honest, so I'm glad you're with us to ensure everything runs perfectly for those that uses IIS.

I don't think your change will break anything and as long as the unit tests passes I see no problem in releasing your fix. However we might need to check all the references in the router classes to ensure that urldecode is not called twice and change your operator check so it's compatible with php 5.6.

Great work! I'll add this to the next release and keep you posted.

/ Simon

[jatubio]

It is always a pleasure to help :)

I understand how difficult it is to test a library like this on all possible servers.

Since I use IIS, I will be happy to help you with the tests and documentation regarding IIS. I added the configuration for IIS to the read.me file :)

[jatubio]

After decode the url, I need to remove accents from some parameters.

If I add this line to __construct() method of Peece\Http\Request class:
$this->uri=removeacents($this->uri);

Like that:

$this->uri = urldecode($this->getHeader('unencoded_url')) ?: $this->getHeader('request-uri');
$this->uri=removeacents($this->uri);

It works ok and for http://www.misite.com/especialidad/Dermatolog%C3%ADa or http://www.misite.com/especialidad/Dermatología I get http://www.misite.com/especialidad/Dermatologia

But, if i don't add these line and try to do this:
$parameter=removeacents(request()->getLoadedRoute()->getParameters['especialidad']) i'm getting the word 'Dermatología' oddly coded.

¿This it's because something is changing the uri or parameters code after __construc() method?

[skipperbent]

Urldecode might change the encoding, so you have to remove urldecode from the route classes which uses request->getUri() otherwise they will be decoded twice.

You can see the classes you need to change in the commit to referenced above.

[jatubio]

I have put the same changes on my code, but don't have distinct results.

This is from my log (v.3.3.6):

On Request->__construct():

   $this->getHeader("request-uri"):/cursos/listado/especialidad/Dermatolog韆
   $this->getHeader("unencoded_url"):/cursos/listado/especialidad/Dermatolog%C3%ADa
   urldecode($this->getHeader("unencoded_url")):/cursos/listado/especialidad/Dermatolog铆a

Only Dermatolog铆a seems to work with removeAcents() function. Then, I can't get request-uri, I have to get unencoded_url and on some moment do a urldecode().

If I left this line, I get this results for getParameter():

1th Test:
$this->uri = $this->getHeader('request-uri'); --> /cursos/listado/especialidad/Dermatolog韆
getParameter() --> Dermatolog韆

2th Test:
$this->uri = $this->getHeader('unencoded_url'); --> /cursos/listado/especialidad/Dermatolog%C3%ADa
getParameter() --> Dermatolog胈a

3th Test:
$this->uri = urldecode($this->getHeader('unencoded_url')); --> /cursos/listado/especialidad/Dermatolog铆a
getParameter() --> Dermatolog胈a

As you can see, something strange happens: the parameter is changed on the second and third tests, but not in the first. Then, if I don't do removeaccents() on __construct() method, later it don't work because parameter have changed his codification.

PS: If I left only this line: $this->uri = $this->getHeader('unencoded_url') ?: $this->getHeader('request-uri');, i get Route not found: "/cursos/listado/especialidad/Dermatolog%C3%ADa" with the changes of your referenced commit.

[skipperbent]

Did some digging and it turns out that PHP's parse_url function doesn't work well with unicode characters.

Did some tests and managed to get it working with a custom url parser. I'll update the pull-request shortly so you can see what's changed.

[skipperbent]

The fix is available here and is pending next release:
https://github.com/skipperbent/simple-php-router/pull/271/files

Note the new Uri class and especially the parseUrl method. Also all references to the old php parse_url has been removed.

[jatubio]

I'm testing only the last pull files and is not working ok. I will wait to your next relase to test them :)

[skipperbent]

It should work - the unit tests with your example returns the correctly encoded characters. You might have to update your parameterRegex though.

Try the latest version 3.4.0, please read the release notes before updating as some changes might break your project.

Let me know if you get it working 👍

[jatubio]

Now, with 3.4.0, http://www.site.com/cursos/listado/especialidad/Dermatología will go to default route, but
http://www.site.com/cursos/listado/especialidad/Dermatologia it's working ok and go to correct route.

This is my ['defaultParameterRegex' => '[\(\)\w\p{L}\s-]+'] (I have added '(' and ')' because have some words with them.

I will try to pass your tests. Last time I tried don't know how to setup phpunit correctly, but now I will try again and tell you.

[skipperbent]

Setting up phpunit is easy. You just run composer install in the simple-php-router directory, then PhpStorm should detect everything automatically.

Sounds like you should consider using querystrings instead as they are guaranteed to support all characters. If you send unicode characters via the path - browser might send them differently to your server.

This unit-tests passes - and the parameters returned are correct.

public function testUnicodeCharacters()
    {
        // Test spanish characters
        TestRouter::get('/cursos/listado/{listado?}/{category?}', 'DummyController@method1', ['defaultParameterRegex' => '[\w\p{L}\s-\í]+']);
        TestRouter::debugNoReset('/cursos/listado/especialidad/cirugía local', 'get');
        $this->assertEquals('/cursos/listado/{listado?}/{category?}/', TestRouter::router()->getRequest()->getLoadedRoute()->getUrl());

        // Test danish characters
        TestRouter::get('/kategori/økse', 'DummyController@method1', ['defaultParameterRegex' => '[\w\ø]+']);
        TestRouter::debugNoReset('/kategori/økse', 'get');
        $this->assertEquals('/kategori/økse/', TestRouter::router()->getRequest()->getLoadedRoute()->getUrl());

        TestRouter::get('/test/{param}', 'DummyController@method1', ['defaultParameterRegex' => '[\w\p{L}\s-\í]+']);
        TestRouter::debugNoReset('/test/Dermatología');

        $parameters = TestRouter::request()->getLoadedRoute()->getParameters();

        $this->assertEquals('Dermatología', $parameters['param']);

        TestRouter::router()->reset();
    }