Steps to reproduce:

1. Create the following content structure:

Home
	Tests (folder)

2. Create a "/mytest1" URL alias for "/Tests" folder;
3. Create a "/mytest1/mytest2" URL alias again for "/Tests" folder;
4. At this stage, both "example.com/mytest1" and "example.com/mytest1/mytest2" will re-direct to "example.com/Tests";
5. Delete the "/mytest1" alias from "/Tests" folder;
6. "example.com/mytest1" now leads to the start page, instead of throwing a 404 error, as would be expected. "example.com/mytest1/mytest2" should also throw a 404 error, but it still leads to "example.com/Tests".