Bug #11156
Conversion fails if source folder is a symlink
100%
Related issues
History
#1 Updated by Alexandru Lungu 9 months ago
If I attempt to convert a project (e.g. Hotel), but the ./abl folder is a symlink, then the conversion fails with:
[java] EXPRESSION EXECUTION ERROR:
[java] ---------------------------
[java] throwException(err)
[java] ^ { Invalid basepath! basepath = ./abl; pkg = abl; [BLOCK id <360777252865> 0:0] }
[java] ---------------------------
[java] EXPRESSION EXECUTION ERROR:
[java] ---------------------------
[java] relpkg = execLib("rel_pkg_path")
[java] ^ { Expression execution error @1:1 }
I had another customer project fail exactly the same, but with different error feedback (#10981).
#3 Updated by Dănuț Filimon 9 months ago
- Status changed from New to WIP
- Assignee set to Dănuț Filimon
I'll pick this one up.
#4 Updated by Dănuț Filimon 9 months ago
There are no issues when the symlink points to a folder in the ~/gcd/hotel_gui directory, but the problem appears when it is part of another folder that is outside ~/gcd/hotel_gui.
This is how normalizeFilename() works right now when the symlink is../abl:
- home = /home/ddf/gcd/hotel_gui and filename = abl/activate-rooms-dialog.w
- abnormal.getCanonicalPath() = /home/ddf/gcd/abl/activate-rooms-dialog.w
- new File(home).getCanonicalPath() = /home/ddf/gcd/hotel_gui
- absolute.startsWith(home) = false
- filename is returned
The issue? abnormal.getCanonicalPath() = /home/ddf/gcd/abl/activate-rooms-dialog.w follows the symlink target instead of preserving it.
- home = /home/ddf/gcd/hotel_gui and filename = abl/activate-rooms-dialog.w
- Path homeRoot = Paths.get(home).toAbsolutePath().normalize();
- Path inputFilename = Path input = Paths.get(filename);
- normalize the inputFilename
- inputFilename.startsWith(homeRoot) => relativize(inputFilename) and return "." + File.separator + inputFilename
- return the filename if nothing happened
I ran a Hotel_GUI conversion and it worked properly, I will commit the changes soon.
#5 Updated by Dănuț Filimon 9 months ago
- % Done changed from 0 to 100
- Status changed from WIP to Review
- reviewer Alexandru Lungu added
Committed 11156a/16373. When using a symlink, make sure the target is preserved instead of followed. (refs: #11156)
Alexandru, please review.
#6 Updated by Dănuț Filimon about 1 month ago
- Related to Bug #11867: Covnersion fails if the abl folder is symlinked added
#7 Updated by Dănuț Filimon about 1 month ago
Rebased 11156a to latest trunk/16752, the branch is now at revision 16753.
#8 Updated by Alexandru Lungu 22 days ago
3 MAJOR, all in normalizeFilename, all from dropping canonicalization:
- Uncanonicalized home breaks the ticket's own scenario. SymbolResolver.pkgExists() feeds canonical (symlink-resolved) paths from exists() into the two-arg overload whose home is now lexical-only. With basepath="./abl" or oo-skeleton-path="./abl/skeleton" symlinked, the containment check fails and a dotted filesystem path lands in the normalized-path annotation. The challenger found basepath="./abl" and oo-skeleton-path="./abl/skeleton" are real configured values in metacom-family projects, and that p2j is a symlink in 5 of 5 configured projects. Hotel_GUI — which uses a real ./skeleton/ dir and has little OO code — plausibly explains why this wasn't caught.
- Relative filenames now resolve against home, not CWD. pkgExists() line 10179 passes an already-relative "./abl/src/..." into normalizeFilename(basepath, ...), producing <cwd>/abl/src/abl/src/.... It still passes startsWith, so the basepath is silently never stripped. Verified with a compiled old-vs-new harness.
- "." → "./" for the project root. relativize() returns an empty path, so Configuration.withFileProfile()'s ".".equals(path) sentinel stops firing and startsWith("./") then matches every artifact — multi-profile conversions get the wrong profile.
#9 Updated by Dănuț Filimon 22 days ago
- Status changed from Review to WIP
- % Done changed from 100 to 70
I will address #11156-8
#10 Updated by Dănuț Filimon 21 days ago
- Status changed from WIP to Review
- % Done changed from 70 to 100
Committed 11156a/16753 to fix #11156-8.
I've tested abl/skeleton folder as symlinks to path outside of the current working directory/in the current directory. I also ended up testing two profiles (both with default="false") and ran a conversion only to find out it is not working, I've added a small fix with that to ensure this doesn't pop up in another task (it is unrelated to the path fixes).
Alexandru, please review.
#11 Updated by Alexandru Lungu 21 days ago
- % Done changed from 100 to 90
- Status changed from Review to WIP
Findings:
1. Major, BuiltinVariable.getValue: {&FILE-NAME} still follows the symlink before calling normalizeFilename() (it uses getCanonicalPath()). With ./abl pointing at /data/src, the result is "src/x.p" instead of "x.p", and that wrong value ends up in the converted code. This isn't new, but it's exactly the layout the ticket is meant to support. The fix is to pass getAbsolutePath() instead, and to check that the name starts with basepath before cutting it off.2. Minor, FileScope.resolveFileName: include files are still resolved by following the symlink, so they get registered under the target path. Files from the file list now keep the ./abl/... name. The call graph then logs "creating a new node for the include file ..." and creates a duplicate node for each include.
3. Minor, Configuration.withFileProfile:
- This older problem only matters now: the argument meant as the legacy case-sensitivity flag receives isJarMode() instead.
- Before this branch the result was thrown away and the code crashed with a NullPointerException right after, so the wrong flag had no effect.
- Now the collection picks the profile. With case-sensitive=true, file-set filters match without regard to case and can pick the wrong profile.
4. Minor, ArtifactManager.normalizeFilename: on Windows only, if a propath entry in p2j.cfg.xml is spelled with different case than the folder on disk, the PROPATH fallback in withFileProfile silently fails to match, and the global config is used instead. The old code corrected the case from disk; the new first pass keeps the spelling it was given.
5. Minor, style: the history entry in the ArtifactManager header says "there is not match" instead of "there is no match".
Please assess which of the above are actual issues as fix.
#12 Updated by Dănuț Filimon 21 days ago
Alexandru Lungu wrote:
Findings:
1. Major, BuiltinVariable.getValue: {&FILE-NAME} still follows the symlink before calling normalizeFilename() (it uses getCanonicalPath()). With ./abl pointing at /data/src, the result is "src/x.p" instead of "x.p", and that wrong value ends up in the converted code. This isn't new, but it's exactly the layout the ticket is meant to support. The fix is to pass getAbsolutePath() instead, and to check that the name starts with basepath before cutting it off.
This one especially is concerning, I found more than 1 call of normalizeFilename() that uses getCanonicalPath(). ALl of those should be fixed to avoid this in the future and have them use getAbsolutePath() as mentioned.
#13 Updated by Dănuț Filimon 20 days ago
- File 11156a.patch
added
I've made several changes, I took a closer look into BuiltinVariable.getValue(), if the normalizedFIlename() is ./abl_src and the base is ./abl, I'd get a trimmed _src/<filename>. Small fix is to normalize the base.
Both normalizeFilename() and addArtifact() use getCanonicalPath() in the project. In SymbolResolver.exists() I made changes because of #11156-8, but I am not sure if:
if (!caseSens)
{
// TODO: this is broken!!!! It was found to fail when the exact case-sensitive match
// actually existed (resolved by checking for that first, above). It is not
// known if it fails in other cases too.
return FileSystemDaemon.getCaseInsensitiveMatch(file, true);
}
should also use getAbsolutePath(), mostly because it is broken as the comment suggests.
More on the MAJOR issue. I checked with a {&FILE-NAME} pointed to data/src and the output was value.p (the name of the file converted), which looks right.
For number 4, it will take more time for me to test it. I am attaching a patch for now.
#14 Updated by Dănuț Filimon 12 days ago
- Status changed from WIP to Review
- % Done changed from 90 to 100
Committed 11156a/16756-16757 for review. Windows testing went fine, even after changing the propath as mentioned in #11156-11.