FWD » Core Development » User Interface

Some initial background:

Eugenie,

I've stumbled upon this error, too... I didn't find why this was happening, the workaround was to >start the P2J server from Eclipse, and not via the server.cmd script. I've also tried running the >server.cmd from a cmd.exe with admin privileges, and no difference...

Thanks,
Constantin

On 02/20/2017 09:30 PM, Eugenie V. Lyzenko wrote:

Ovidiu,

I'm checking the instructions to set up FWD server and client on Windows and need some assistance from someone who has running both server and web client on Windows. Do you have this working config, right?

If so can you comment please what is wrong in my config. The issue happening when I try to use Web client(both Mozilla and IE) on Windows - after entering User/Password into login screen. The server.log is:

---------
ERROR: API        = CreateProcessWithLogonW
       error code = -2147024809
       message    = The parameter is incorrect.
[02/20/2017 22:14:50 PST] (ClientSpawner.spawn():SEVERE) {qtp421077315-45} Failed spawn with exit code: -6
---------

The message in browser is: "Can not create process for user 'evl'". Because the swing client is working I think it looks like the user config issue. But I check all I think related and have not found the answer.

Regards,
Eugenie.

Eugenie,

I also had some issues starting the spawner. I stopped working on a solution for windows and I used a >mixed approach: server on Linux and connecting with clients remotely, from Windows. This worked for >all 3 client types. Firefox and Edge browsers went smooth, but IE has some issues.
I did not try to launch the server from IDE. If that helps I will try too.

Thanks,
Ovidiu

The system the issue was detected and confirmed:
Windows 7 Ultimate, 64-bit, Oracle JDK 64-bit, running inside Ubuntu 16.04 KVM. The run mode for server: using server.cmd.

The options the CreateProcessWithLogonW is trying to execute:

User           = "valid user" 
Password       = "valid password" 
Command        = java -Xmx512m -Djava.awt.headless=true -Djava.library.path=c:\opt\poc\p2j\build\lib -classpath c:\opt\spawn\p2j.jar com.goldencode.p2j.main.ClientDriver server:spawner:uuid=a52b3320-2e8f-4472-a79b-ad2796e832e0 net:server:host=localhost net:connection:secure=true net:server:secure_port=3333 client:driver:type=gui_web client:gui:desktopHeight=768 client:gui:desktopWidth=1024 client:gui:taskbar=true client:gui:graphicsCached=true client:web:socketTimeout=-1 client:web:watchdogTimeout=-1 client:web:maxBinaryMessage=32768 client:web:maxTextMessage=4096 client:web:maxIdleTime=90000 client:web:embedded=false client:web:port=0 client:web:host=localhost web:referrer:url=https://localhost:7443/gui client:cmd-line-option:debugalert=true

Working dir    = c:\opt\poc\deploy\client

java is on the Path, directories exist. Planning to check ability to have the same user for server and client, correct user string format, may be firewall settings or STARTUPINFO fields. Also the command line must be writeable memory region.

So continue debugging.

Some investigation result:
1. The first thing that is wrong is the environment passing to CreateProcessWithLogonW API. It contains no valid variables. That's why the error tells about The parameter is incorrect.

The fix for this is to change from

   if (!_CreateEnvironmentBlock(&lpvEnv, hToken, TRUE))

   if (!_CreateEnvironmentBlock(&lpvEnv, hToken, FALSE))

This way we are getting the reasonable Windows environment.

2. However it does not fix the main issue. The call CreateProcessWithLogonW is executing without error but the spawned process is not starting with error inside browser: "Can not start embedded server".

3. Attempts to change CreateProcessWithLogonW to CreateProcessWithTokenW and use user token obtained while login call is not working too with message A required privilege is not held by the client.

4. I think it is a kind of virus or malware protection to suppress executing everything on every system with known login. Need additional investigation for more details. In the Web I saw the CreateProcessWithLogonW is not properly working on Windows 7.

5. But there is a solution(at least for now until we find something better). The function CreateProcessW works fine with java command. The web server/client are working fine on the same Windows machine for different user accounts. The only requirement - the user must exist on the Windows system. The administrator access is not required to run web client.

6. On the other hand I think we do not actually need the CreateProcessWithLogonW because the user login is verified before running the client code. If user is wrong or password is wrong the client the process is just not creating(no matter which function is used to create process). The usage of the CreateProcessW give us more compatibility with different Windows systems.

So if no objection I will change to process creation and commit the changes into 3209e.

We spawn programs in 2 modes:

1. End-User Authenticated

In this mode, there is a real user that either does or does not have a valid Windows account. If they have a valid account, they enter that information in the logon panel of the web client and the spawning occurs using that userid and password.

2. Trusted

This mode is used for the embedded web client AND also for appserver and batch clients. In this mode, there is no valid account that is given to us by the user. In the appserver and batch cases, there isn't even an end user there because they are non-interactive. Instead, the FWD server is configured to use a specific account and the spawner MUST have the administrative rights to start a child process in that account. There will be no logon and there is no password to provide.

But there is a solution(at least for now until we find something better). The function CreateProcessW works fine with java command. The web server/client are working fine on the same Windows machine for different user accounts. The only requirement - the user must exist on the Windows system. The administrator access is not required to run web client.

I don't think this is correct. It runs in the context of the calling process, so every user would have the same account as the spawner. While it is possible that some use cases might be OK, it isn't honoring the end user's credentials in End User Authenticated mode so it is not correct.

It also cannot be used for Trusted mode, so it is not a solution.

6. On the other hand I think we do not actually need the CreateProcessWithLogonW because the user login is verified before running the client code. If user is wrong or password is wrong the client the process is just not creating(no matter which function is used to create process). The usage of the CreateProcessW give us more compatibility with different Windows systems.

This should work to assign the given process the right context if there is the correct credentials. It is supposed to be supported from Windows XP/Windows Server 2003 and later. But there is a comment that it is for desktop apps only, so I wonder if that is part of the problem.

I think the other approach to try is CreateProcessAsUser. I think the core issue here is that the spawner needs administrative rights for either of these approaches to work.

From the Windows API reference:

The CreateProcessWithLogonW requires no special privileges, but the specified user account must be allowed to log on interactively. Generally, it is best to use CreateProcessWithLogonW to create a process with alternate credentials.

So CreateProcessWithLogonW can be used without admin rights, but only for the End User Authenticated mode. If we don't have a password to use, then I think we need the spawner to have admin rights to allow this to work for Trusted mode.

I think calling CreateProcessWithTokenW and doing the LogonUser call to get the token is the equivalent of CreateProcessWithLogonW. It will work for End User Authenticated mode but probably not for Trusted mode.

Greg Shah wrote:

2. Trusted

This mode is used for the embedded web client AND also for appserver and batch clients. In this mode, there is no valid account that is given to us by the user. In the appserver and batch cases, there isn't even an end user there because they are non-interactive. Instead, the FWD server is configured to use a specific account and the spawner MUST have the administrative rights to start a child process in that account. There will be no logon and there is no password to provide.

But there is a solution(at least for now until we find something better). The function CreateProcessW works fine with java command. The web server/client are working fine on the same Windows machine for different user accounts. The only requirement - the user must exist on the Windows system. The administrator access is not required to run web client.

I don't think this is correct. It runs in the context of the calling process, so every user would have the same account as the spawner. While it is possible that some use cases might be OK, it isn't honoring the end user's credentials in End User Authenticated mode so it is not correct.

It also cannot be used for Trusted mode, so it is not a solution.

Understood. From the winspan.c module this means no log-in case(the spawn has two working mode), correct? But for this case(Trusted mode, no user ID/password) the current implementation uses CreateProcessW to start the java code. Does it mean this is also not very good and we have to find out alternative approach here too?

I think I was unclear.

The documentation for CreateProcessW states this:

Creates a new process and its primary thread. The new process runs in the security context of the calling process.

This suggests that CreateProcessW cannot be used for either of our use cases. The End User Authentication mode deliberately provides a userid and password that cannot be used here.

But even the Trusted mode cannot be supported (except in the one use case where the spawner's account is the same as the account for every user, which is a rare and not very useful case). In other words, even in Trusted mode, we must be able to force the child process to be executed in the context of a specific account. In most cases, that account will be different from the spawner's account, which means CreateProcessW cannot be used.

Greg Shah wrote:

I think I was unclear.

The documentation for CreateProcessW states this:

Creates a new process and its primary thread. The new process runs in the security context of the calling process.

This suggests that CreateProcessW cannot be used for either of our use cases. The End User Authentication mode deliberately provides a userid and password that cannot be used here.

But even the Trusted mode cannot be supported (except in the one use case where the spawner's account is the same as the account for every user, which is a rare and not very useful case). In other words, even in Trusted mode, we must be able to force the child process to be executed in the context of a specific account. In most cases, that account will be different from the spawner's account, which means CreateProcessW cannot be used.

I see. The API call CreateProcessWithLogonW is the best one that fits our requirements here. It is still unclear why it does not work executing from browser context.

I've made the experiments. Looks like my previous assumption about environment change in CreateEnvironmentBlock was wrong. We need to inherit one from calling process(las parameter to become TRUE in CreateEnvironmentBlock) because we use the environment to pass encrypted user/password to the starting client driver, correct? The client driver extracting them by(ClientCore)

System.getenv(TemporaryAccount.P2J_SUBJECT);
...

...
bc.setConfigItem("access", "subject", "id", TemporaryAccount.fromHex(subject));
...

So we prepare the updated environment before calling spawn in End-User Authenticated mode, then run spawned process with new environment and the started process extracts the values(_SUBJECT/_PASSWORD) from environment. Is it correct understanding?

Another investigation result is: running spawn.exe as standalone application with log-on more calls CreateProcessWithLogonW without errors. This inclines me to thinking the issue is in somewhere in the way we do spawn. Something inside browser is different from the pure command line shell. Adding the user policy to be considered as system(via Windows Control Panel) does not work as well.

Continue investigations.

It is likely that the code in winspawn.c is OK, since it did work in past testing. I think the most likely case is that we have not configured it properly so that it works.

Please read in #1811-497 and beyond to see some of the discussion when we developed the Windows spanwer.

Looks like my previous assumption about environment change in CreateEnvironmentBlock was wrong. We need to inherit one from calling process(las parameter to become TRUE in CreateEnvironmentBlock) because we use the environment to pass encrypted user/password to the starting client driver, correct? The client driver extracting them by(ClientCore)

Yes. In the "temporary P2J server connection" case (used for appserver agents and batch processes), this is how we pass the temporary account credentials to the spawned P2J client.

As noted above, you should assume that the code is correct and look for configuration issues. Since we are seeing permission problems, I suspect we have an rights issue.

So we prepare the updated environment before calling spawn in End-User Authenticated mode, then run spawned process with new environment and the started process extracts the values(_SUBJECT/_PASSWORD) from environment. Is it correct understanding?

I don't think so. The temporary account processing is a 3rd mode, used for the non-interactive client cases (appserver/batch).

I think the userid and password for end user authenticated mode will be passed as command line arguments to spawn.exe.

Because of limits in Windows, I think the trusted mode requires the password for the target account unless the target account is the same as the account running the spawner. I think we store the password for this trusted mode in the directory, but this would only affect the embedded mode case. Still, this may be a configuration issue to fix. Regardless of the end user authenticated/trusted mode, I think the userid and password are passed to the spawner as command line args.

Another investigation result is: running spawn.exe as standalone application with log-on more calls CreateProcessWithLogonW without errors. This inclines me to thinking the issue is in somewhere in the way we do spawn. Something inside browser is different from the pure command line shell.

This is not running in the browser. The browser contacts the FWD server. The FWD server either:

1. Runs the spawner directly as a child process of the FWD server. OR
2. Contacts a "remote launcher", which is a special FWD client process that is connected to the FWD server and is waiting for a request to run the spanwer as a child process. This allows us to separate the launch of a client on a different machine that the one on which the FWD server exists.

Either way we have a Java process running FWD code that is running the spawner as a child process. The browser is not involved at all in the execution of the spawner.

I think the problem is the configuration of how we are invoking the spawner AND/OR the permissions/account setup on the Windows system.

To be clear, please note this text from Marius in #1811-497:

I'm afraid that in Windows OS it's not possible to create a process on another account using only the user name.
We must provide both user name and password. Without credentials we can create processes only for the account on which the server (spawner) is running.

And this from Constantin in #1811-500:

I agree it should be optional, but I don't think is needed to be kept at the account. The system user is read from the i.e. runtime/default/runtime/<account-id>/clientConfig/systemUser path (for both user and process accounts). So, is best to keep it with the system user, in clientConfig/systemPassword node.

I think this is in regard to configuring the password for the account to be used.

Constantin: please review this task and comment with any ideas.

One small issue with spawn_logon() is that when using CreateEnvironmentBlock, we are supposed to call DeleteEnvironmentBlock after the call to CreateProcessWithLogonW so that the memory can be released.

Eugenie: Have you read Spawner Setup and Configuration? Perhaps we just need the encoded password in the directory and to use the spawn.ini to store the encryption key.

Greg Shah wrote:

Constantin: please review this task and comment with any ideas.

What bugs me is when the server was started from Eclipse using the exact same command as the one generated by server.cmd, then everything worked fine. I don't know what the difference at the OS level is in these two cases.

I wonder if is not a change in some Win7 update which causes us this issue; I'm not saying that we don't need to solve it, but maybe is worth it to check with a non-updated win7 installation - just the pristine ISO.

Greg Shah wrote:

Eugenie: Have you read Spawner Setup and Configuration? Perhaps we just need the encoded password in the directory and to use the spawn.ini to store the encryption key.

I have tried to turn it on:
1. Generate encrypted for my password.
2. Ckange encrypted to 1 in spawn.ini
3. Add systemUser, systemPassword to clientConfig section of the directory.xml.

No results. I think because I use POC application to test - the login and password is entering into browser's pane.

I have tried even more. To run the server as one user and run/login the client as another. The result is not better. The winspawn module fails in CreateEnvironmentBlock with error code 5(Access denied).

Yes, I think something is wrong/incomplete in user accounts/security settings.

Constantin Asofiei wrote:

Greg Shah wrote:

Constantin: please review this task and comment with any ideas.

What bugs me is when the server was started from Eclipse using the exact same command as the one generated by server.cmd, then everything worked fine. I don't know what the difference at the OS level is in these two cases.

I wonder if is not a change in some Win7 update which causes us this issue; I'm not saying that we don't need to solve it, but maybe is worth it to check with a non-updated win7 installation - just the pristine ISO.

Yes, may be some system update changed this(I'm keeping my Win 7 systems up to date).

The question is: Do you have now the server/web client working on current Windows(with Eclipse at least as you noted)?

One more point.

If this is the Windows update interference - I think we need to implement the approach that is not dependent on MS update strategy(and we also can not direct the customer to turn the update off).

Eugenie Lyzenko wrote:

The question is: Do you have now the server/web client working on current Windows(with Eclipse at least as you noted)?

Yes, when server is ran from Eclipse, embedded and web mode both work; when ran from server.cmd, they don't. I can't see anything relevant in the event viewer logs - did you check them?

If this is the Windows update interference - I think we need to implement the approach that is not dependent on MS update strategy(and we also can not direct the customer to turn the update off).

The idea was maybe if we identify what caused this regression (as at some point these used to work) then it might lead us to determine a solution for it.

Constantin Asofiei wrote:

If this is the Windows update interference - I think we need to implement the approach that is not dependent on MS update strategy(and we also can not direct the customer to turn the update off).

The idea was maybe if we identify what caused this regression (as at some point these used to work) then it might lead us to determine a solution for it.

I have just installed:
1. Clean Windows XP 64-bit version(SP 2).
2. POC application

The result is the same as for Windows 7:

ERROR: API        = CreateProcessWithLogonW
       error code = -2147024809
       message    = The parameter is incorrect.

So I guess it is not from Windows 7 update. May be the server for web client was ever started via CMD file in Windows XP or 7? May be Windows 8 was used to test?

What is so special in running within Eclipse?

Marius initially wrote the winspawn.c on Windows 7 64-bit in March 2014. It was working at that time.

I agree that running the FWD server from Eclipse seems to be the key part of the investigation. I don't know what could be different. What account and rights does Eclipse have when running?

Eugenie: it is worth having you get this Eclipse launched server going on your system to dig deeper.

Greg Shah wrote:

Marius initially wrote the winspawn.c on Windows 7 64-bit in March 2014. It was working at that time.

I agree that running the FWD server from Eclipse seems to be the key part of the investigation. I don't know what could be different. What account and rights does Eclipse have when running?

Eugenie: it is worth having you get this Eclipse launched server going on your system to dig deeper.

Yes, I think about this too. Do I have to use eclipse_gcd_custom_luna_distribution_20150605.zip from our shared directory as noted in the docs? Planning to set up it on my Windows KVM guest.

Constantin, do you use direct java classes loading to start the server within Eclipse.

Eugenie Lyzenko wrote:

Constantin, do you use direct java classes loading to start the server within Eclipse.

1. no source folders
2. add all jars from build/lib (p2j.jar, hotel.jar, etc)
3. add a run configuration with target directory: deploy/server and add the JVM params and application params found in the server.cmd

Guys, have a good news.

Finally I have got the web client running for server started via CMD. You do not beleive but the issue was the server.cmd script. The environment variables introduced there conflict with normal CreateProcessWithLogonW or CreateEnvironmentBlock. So when I distilled everything but real CMD command the web client started. The command file that works:

:: minimalistic style standard P2J server startup script
@echo off
CMD.EXE /C java -Xmx768m -server -classpath .;..\lib\<CUSTOMER_CONVERTED_APPLICATION_JAR_HERE>.jar;..\lib\akiban-persistit-3.3.0.jar;..\lib\antlr-2.7.7.jar;..\lib\asm-5.0.3.jar;..\lib\aspectjrt-1.8.10.jar;..\lib\axiom-api-1.2.13.jar;..\lib\axiom-impl-1.2.13.jar;..\lib\axis2-adb-1.6.2.jar;..\lib\axis2-kernel-1.6.2.jar;..\lib\axis2-transport-http-1.6.2.jar;..\lib\axis2-transport-local-1.6.2.jar;..\lib\batik-anim-1.8.jar;..\lib\batik-awt-util-1.8.jar;..\lib\batik-bridge-1.8.jar;..\lib\batik-codec-1.8.jar;..\lib\batik-css-1.8.jar;..\lib\batik-dom-1.8.jar;..\lib\batik-ext-1.8.jar;..\lib\batik-extension-1.8.jar;..\lib\batik-gui-util-1.8.jar;..\lib\batik-gvt-1.8.jar;..\lib\batik-parser-1.8.jar;..\lib\batik-script-1.8.jar;..\lib\batik-svg-dom-1.8.jar;..\lib\batik-svggen-1.8.jar;..\lib\batik-swing-1.8.jar;..\lib\batik-transcoder-1.8.jar;..\lib\batik-util-1.8.jar;..\lib\batik-xml-1.8.jar;..\lib\bcel-5.2.jar;..\lib\bcpkix-jdk15on-1.50.jar;..\lib\bcprov-jdk15on-1.50.jar;..\lib\blueprints-core-2.4.0.jar;..\lib\c3p0-0.9.5.2.jar;..\lib\cglib-2.1_3.jar;..\lib\colt-1.2.0.jar;..\lib\common-image-3.1.2.jar;..\lib\common-io-3.1.2.jar;..\lib\common-lang-3.1.2.jar;..\lib\commons-beanutils-core-1.7.0.jar;..\lib\commons-codec-1.7.jar;..\lib\commons-collections-3.2.2.jar;..\lib\commons-configuration-1.6.jar;..\lib\commons-httpclient-3.1.jar;..\lib\commons-lang-2.5.jar;..\lib\commons-logging-1.1.1.jar;..\lib\dom4j-1.6.jar;..\lib\ehcache-core-2.6.2.jar;..\lib\fwd-h2-1.4.192.jar;..\lib\fwd-hibernate-core-4.1.8.jar;..\lib\guava-14.0.1.jar;..\lib\hibernate-c3p0-4.1.8.Final.jar;..\lib\hibernate-commons-annotations-4.0.1.Final.jar;..\lib\hibernate-ehcache-4.1.8.Final.jar;..\lib\hibernate-jpa-2.0-api-1.0.1.Final.jar;..\lib\high-scale-lib-1.1.2.jar;..\lib\hppc-0.4.2.jar;..\lib\httpcore-4.0.jar;..\lib\imageio-bmp-3.1.2.jar;..\lib\imageio-core-3.1.2.jar;..\lib\imageio-metadata-3.1.2.jar;..\lib\javassist-3.15.0-GA.jar;..\lib\javax.servlet-api-3.1.0.jar;..\lib\jboss-logging-3.1.0.GA.jar;..\lib\jboss-transaction-api_1.1_spec-1.0.0.Final.jar;..\lib\jetty-all-9.3.8.v20160314-uber.jar;..\lib\jetty-proxy-9.3.8.v20160314.jar;..\lib\jta-1.1.jar;..\lib\kryo-2.21-shaded.jar;..\lib\lucene-analyzers-common-4.4.0.jar;..\lib\lucene-core-4.4.0.jar;..\lib\lucene-queries-4.4.0.jar;..\lib\lucene-queryparser-4.4.0.jar;..\lib\lucene-spatial-4.4.0.jar;..\lib\mail-1.4.jar;..\lib\mchange-commons-java-0.2.11.jar;..\lib\metrics-core-3.0.0-BETA3.jar;..\lib\msv-core-2010.2.jar;..\lib\neethi-3.0.2.jar;..\lib\p2j.jar;..\lib\p2jadmin.jar;..\lib\p2jpl.jar;..\lib\p2jspi.jar;..\lib\postgresql-9.4.1211.jar;..\lib\quartz-2.2.1.jar;..\lib\relaxngDatatype-20020414.jar;..\lib\slf4j-api-1.7.5.jar;..\lib\slf4j-jdk14-1.6.1.jar;..\lib\spatial4j-0.3.jar;..\lib\stax-utils-20070216.jar;..\lib\stax2-api-3.1.1.jar;..\lib\titan-core-0.4.2.jar;..\lib\titan-lucene-0.4.2.jar;..\lib\titan-persistit-0.4.2.jar;..\lib\woodstox-core-asl-4.2.0.jar;..\lib\wsdl4j-1.6.2.jar;..\lib\xml-apis-1.3.04.jar;..\lib\xml-apis-ext-1.3.04.jar;..\lib\xmlgraphics-commons-2.0.jar;..\lib\XmlSchema-1.4.7.jar;..\lib\xsdlib-2010.1.jar -Djava.system.class.loader=com.goldencode.p2j.classloader.MultiClassLoader com.goldencode.p2j.main.ServerDriver server.xml net:connection:secure=true net:server:secure_port=3333 net:server:insecure_port=3433 1>stdout.log 2>server.log

I have not tested with Eclipse but thanks to Constantin I've got idea for what can be wrong and what I can do in a shorter time(I never used Eclipse and as I can see need additional time to learn how it works).

So I have made only one change to winspan.c for adding DestroyEnvironmentBlock call to cleanup environment block on exit. If no objections I will upload it for review int 3209e.

And certainly we need to fix deploy/server/server.cmd because original file does not work and one above has no configurable options so is not acceptable too. So I'm planning to cleanup original server.cmd if no objections.

This is the modified deploy/server/server.cmd file that works and keeps all functionality of the old one.

:: standard P2J server startup script
@echo off
setlocal enableextensions enabledelayedexpansion

:: defaults
set cpath=.
for %%f in (..\lib\*.jar) do call :add_jar %%f
set debug=0
set suspend=n
set test=n
set hprof=
set prog=java
set srvr=-server
set heap=768
set outlog=stdout.log
set errlog=server.log
set cfg=server.xml
set port=
set portbase=3333
set portstep=100
set mode=
set statcode=n
set instance=0

set usage[0]=Usage: %0 [-dyptksw1] [-z cp] [-h heap] [-l log] [-c cfg] [-i inst]
set usage[1]=Where:
set usage[2]=-d       = enable JVM debug mode (debugger port will be set as 2080 + instance #)
set usage[3]=-y       = in JVM debug mode enable, suspend the JVM on startup (does not suspend by default)
set usage[4]=-p       = enable JVM hprof (profiling output to gc.log)
set usage[5]=-t       = test mode (displays the command but doesn't execute)
set usage[6]=-k       = kill mode (shuts down the specified/default instance)
set usage[7]=-s       = status mode (displays the status of the specified/default instance)
set usage[8]=-w       = wait mode (does not return until the specified/default instance is STATUS_RUNNING)
set usage[9]=-1       = enable C1 HotSpot compiler (client compiler for the JVM)
set usage[10]=-z      = override the classpath which would otherwise be set to %cpath%
set usage[11]=-h      = set JVM max heap size (must be an integer number of MB)
set usage[12]=-l      = log file name (all STDERR is redirected to this file, defaults to ./server.log)
set usage[13]=-c      = config file name (P2J bootstrap config file name, defaults to server.xml)
set usage[14]=-i      = server instance number (0..9, assigns the P2J server port, defaults to 0)

set argCount=0
for %%x in (%*) do (
   set /A argCount=!argCount!+1
   set "argVec[!argCount!]=%%~x" 
)
set srvr=-server
set /a i=1
:while1
   set carg=!argVec[%i%]!

   if "%carg%" equ "-d" (
      set debug=1
   ) else if "%carg%" equ "-y" (
      set suspend=y
   ) else if "%carg%" equ "-p" (
      set hprof=-Xrunhprof:heap=dump,doe=y,format=b,file=gc.log
   ) else if "%carg%" equ "-t" (
      set prog=echo %prog% 
      set test=y
   ) else if "%carg%" equ "-k" (
      set mode=-k
      set statcode=y 
   ) else if "%carg%" equ "-s" (
      set mode=-s
      set statcode=y
   ) else if "%carg%" equ "-w" (
      set mode=-w -1
      set statcode=y 
   ) else if "%carg%" equ "-1" (
      set srvr=
   ) else if "%carg%" equ "-z" (
      set /A "i=i+1" 
      call :setvar cpath !i!
   ) else if "%carg%" equ "-h" (
      set /A "i=i+1" 
      call :setvar heap !i!
   ) else if "%carg%" equ "-l" (
      set /A "i=i+1" 
      call :setvar log !i!
   ) else if "%carg%" equ "-c" (
      set /A "i=i+1" 
      call :setvar cfg !i!
   ) else if "%carg%" equ "-i" (
      set /A "i=i+1" 
      call :setvar instance !i!
   ) else if "%carg%" equ "help" (
      for /L %%a in (0,1,14) do call echo.%%usage[%%a]%%
   )
   set /a i=%i%+1
   if %i% leq %argCount% ( goto :while1 )

:: final arg prep
set cpath=-classpath %cpath%
set /A dport=(2080 + %instance%)
set /A sport=(%portbase% + %instance%)
set /A iport=(%portbase% + %instance% + %portstep%)
set iport=net:server:insecure_port=%iport%
set port=net:connection:secure=true net:server:secure_port=%sport% %iport%

if %debug% equ 1 ( set dtxt=-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=%dport%,server=y,suspend=%suspend% ) else ( set dtxt= )

set maxheap=-Xmx%heap%m

:: prepare command line to run the server
set serverCmd=%prog% %maxheap% %srvr% %dtxt% %cpath% -Djava.system.class.loader=com.goldencode.p2j.classloader.MultiClassLoader com.goldencode.p2j.main.ServerDriver %mode% %cfg% %port%
:: long lines protection 
if "%test%" equ "y" (
   set serverCmd="%serverCmd%
) else (
   set serverCmd="%serverCmd% 1>%outlog% 2>%errlog%" 
)

:: clean up environment variables no longer needed
set usage[0]=
set usage[1]=
set usage[2]=
set usage[3]=
set usage[4]=
set usage[5]=
set usage[6]=
set usage[7]=
set usage[8]=
set usage[9]=
set usage[10]=
set usage[11]=
set usage[12]=
set usage[13]=
set usage[14]=
set dport=
set sport=
set iport=
set portbase=
set portstep=
set instance=
set i=
set argCount=
set carg=
set debug=
set dport=
set heap=
set suspend=
set prog=
set maxheap=
set srvr=
set dtxt=
set cpath=
set mode=
set cfg=
set port=
set outlog=
set errlog=

:: run the P2J server
CMD.EXE /C %serverCmd%

set rc=%errorlevel%

if "%statcode%" equ "y" (
   switch %rc%
   case 0
      set status=STATUS_RUNNING
   case 1
      set status=STATUS_TIMEOUT
   case 2
      set status=STATUS_UNKNOWN
   case 3
      set status=STATUS_STOPPED
   endswitch

   echo Result = %status%
fi

endlocal

goto:eof

:add_jar
set cpath=%cpath%;%1
goto:eof

:setvar
set k=%2
set %1=!argVec[%k%]!
goto:eof

:getPID  [RtnVar]
::
:: Store the Process ID (PID) of the currently running script in environment variable RtnVar.
:: If called without any argument, then simply write the PID to stdout.
::
setlocal disableDelayedExpansion
:getLock
set "lock=%temp%\%~nx0.%time::=.%.lock" 
set "uid=%lock:\=:b%" 
set "uid=%uid:,=:c%" 
set "uid=%uid:'=:q%" 
set "uid=%uid:_=:u%" 
setlocal enableDelayedExpansion
set "uid=!uid:%%=:p!" 
endlocal & set "uid=%uid%" 
2>nul ( 9>"%lock%" (
  for /f "skip=1" %%A in (
    'wmic process where "name='cmd.exe' and CommandLine like '%%<%uid%>%%'" get ParentProcessID'
  ) do for %%B in (%%A) do set "PID=%%B" 
  (call )
))||goto :getLock
del "%lock%" 2>nul
endlocal & if "%~1" equ "" (echo(%PID%) else set "%~1=%PID%" 
exit /b

Eugenie Lyzenko wrote:

Guys, have a good news.

Finally I have got the web client running for server started via CMD. You do not beleive but the issue was the server.cmd script. The environment variables introduced there conflict with normal CreateProcessWithLogonW or CreateEnvironmentBlock.

Great work, this is really unexpected for me... just curious, any idea which vars caused the problem?

The modified server.cmd in note 25 looks OK. Please fix the server.cmd for Hotel ChUI, too, test it and you can commit both versions. Also, add a comment why the var cleanup is needed.

Constantin Asofiei wrote:

The modified server.cmd in note 25 looks OK. Please fix the server.cmd for Hotel ChUI, too, test it and you can commit both versions. Also, add a comment why the var cleanup is needed.

I have committed the POC repo to the revision 45 with new server.cmd.

Can you tell me where can I find the repo to commit Hotel ChUI version of the script?

Eugenie Lyzenko wrote:

Can you tell me where can I find the repo to commit Hotel ChUI version of the script?

The repos for the Hotel ChUI and Hotel GUI are in ~/secure/code/p2j_repo/samples/hotel and ~/secure/code/p2j_repo/samples/hotel_gui - please fix both.

Constantin Asofiei wrote:

The repos for the Hotel ChUI and Hotel GUI are in ~/secure/code/p2j_repo/samples/hotel and ~/secure/code/p2j_repo/samples/hotel_gui - please fix both.

Done. The Hotel ChUI and Hotel GUI have been updated to revisions 37 and 91 respectively.

I have made only one change to winspan.c for adding DestroyEnvironmentBlock call to cleanup environment block on exit. If no objections I will upload it for review int 3209e.

Yes, please do.

Greg Shah wrote:

I have made only one change to winspan.c for adding DestroyEnvironmentBlock call to cleanup environment block on exit. If no objections I will upload it for review int 3209e.

Yes, please do.

The 3209e updated for review to 11216.

The only change is adding DestroyEnvironmentBlock to clean up allocated resource.

Do we need to change the spawn_nologon() to replace original CreateProcessW() with approach discussed in note #7 above? Or for now we can leave the current code in winspawn.c as is?

Do we need to change the spawn_nologon() to replace original CreateProcessW() with approach discussed in note #7 above?

No. The nologon case is that "rare case" that I was mentioning in #3250-7 which is that the spawned sessions are meant to run in the same user as the server. Generally, that is a bad idea (for security purposes), although I can see it being used in the case of appserver/batch processes. It is still a poor security design, even in that case but we will allow the customer to make the decision.

Or for now we can leave the current code in winspawn.c as is?

Leave it.

Eugenie, I still have problems with running server.cmd on windows with P2J trunk rev 11143 and hotel GUI rev 106 - can you please double-check if it works on your side? I get this on a Windows 7 64 bit:

ERROR: API        = CreateProcessWithLogonW
       error code = 1783
       message    = The stub received bad data.

Constantin Asofiei wrote:

Eugenie, I still have problems with running server.cmd on windows with P2J trunk rev 11143 and hotel GUI rev 106 - can you please double-check if it works on your side? I get this on a Windows 7 64 bit:
[...]

What about POC application? Does it have the same issue on your system?

Are you running the server via server.cmd? Or Eclipse?

Eugenie Lyzenko wrote:

Constantin Asofiei wrote:

Eugenie, I still have problems with running server.cmd on windows with P2J trunk rev 11143 and hotel GUI rev 106 - can you please double-check if it works on your side? I get this on a Windows 7 64 bit:
[...]

What about POC application? Does it have the same issue on your system?

Yes, POC has the same issue.

Are you running the server via server.cmd? Or Eclipse?

Via server.cmd - is the same version in Hotel GUI and POC.

Constantin Asofiei wrote:

Eugenie Lyzenko wrote:

Constantin Asofiei wrote:

Eugenie, I still have problems with running server.cmd on windows with P2J trunk rev 11143 and hotel GUI rev 106 - can you please double-check if it works on your side? I get this on a Windows 7 64 bit:
[...]

What about POC application? Does it have the same issue on your system?

Yes, POC has the same issue.

Are you running the server via server.cmd? Or Eclipse?

Via server.cmd - is the same version in Hotel GUI and POC.

A bit more questions:
1. Do you have the same version as noted on our Wiki page instructions for MinGW(4.9.0) and libffi(3.2.1)? I did my tests for these revisions.
2. Is the server process running under same account as user accessing web client?
3. Both server and client are running on the same machine?
4. What is the user account types? Admin, regular user?
5. Have you ever server/client pair running fine on your system? What was the FWD revision?
6. Please check is the spawn.exe in the spawn directory match one you build in fwd/build/native

I'm running good, from the error background it is possible the service buffer is overflow on your system? Another possible reason is the user access control issue.

Eugenie Lyzenko wrote:

A bit more questions:
1. Do you have the same version as noted on our Wiki page instructions for MinGW(4.9.0) and libffi(3.2.1)? I did my tests for these revisions.

MinGW I think is 4.8.2, libffi I'm not sure - haven't changed this installation since I set it up ~2 years ago.

2. Is the server process running under same account as user accessing web client?

Yes.

3. Both server and client are running on the same machine?

Yes.

4. What is the user account types? Admin, regular user?

The user is an user with Administrator privileges.

5. Have you ever server/client pair running fine on your system? What was the FWD revision?

I can't say 100% sure, but I think it might have worked...

6. Please check is the spawn.exe in the spawn directory match one you build in fwd/build/native

Yes, they are the same.

It is the real system, not virtual machine?

Do you have the possibility to test it with Windows KVM guest inside Ubuntu?

The installation is pretty simple: 1-1.5 hours. But if you will be going to install let me know please, there are the set of magic hints for Windows.

May be there is some adjustment that affects the server/client working. I'll try to identify on clean system.

Eugenie Lyzenko wrote:

It is the real system, not virtual machine?

Yes, is a real system, not a VM.

Do you have the possibility to test it with Windows KVM guest inside Ubuntu?

Before doing this, can you point me the redmine task/wiki which details how to build MINGW and libffi? I want to try the versions you mention first.

May be there is some adjustment that affects the server/client working. I'll try to identify on clean system.

Yes, please go ahead and double check from a clean installation. If it still works, let it go through all the windows updates and see how it behaves then.

Constantin Asofiei wrote:

Do you have the possibility to test it with Windows KVM guest inside Ubuntu?

Before doing this, can you point me the redmine task/wiki which details how to build MINGW and libffi? I want to try the versions you mention first.

You do not need to build them. Just take the 64-bit binaries and use. I can upload the required packages if you tell me the place(on devbox for example).

Eugenie Lyzenko wrote:

Constantin Asofiei wrote:

Do you have the possibility to test it with Windows KVM guest inside Ubuntu?

Before doing this, can you point me the redmine task/wiki which details how to build MINGW and libffi? I want to try the versions you mention first.

You do not need to build them. Just take the 64-bit binaries and use. I can upload the required packages if you tell me the place(on devbox for example).

Pleace them on devsrv01:/tmp/ and I'll pick them from there. And let me know where I need to place them.

They are already available at https://proj.goldencode.com/downloads/

Greg Shah wrote:

They are already available at https://proj.goldencode.com/downloads/

Yes, for 64-bit Windows: mingw64_4.9.0_20140425.zip and libffi_3.2.1_prebuilt_mingw_4.9.0_64bit.zip.

Just unzip and copy all things as documented in: https://proj.goldencode.com/projects/p2j/wiki/Development_Environment_Setup_and_Building_FWD_on_Windows

To build libffi from sources the MSYS is required.

Eugenie: please provide MSYS and I will put it up there too.

Greg Shah wrote:

Eugenie: please provide MSYS and I will put it up there too.

Done. The package is on my hone at devbox, windows subdirectory:

evl@devsrv01:~$ ls windows
MSYS32_20111123.zip

Eugenie Lyzenko wrote:

Just unzip and copy all things as documented in: https://proj.goldencode.com/projects/p2j/wiki/Development_Environment_Setup_and_Building_FWD_on_Windows

Done, but FWD native task fails when building with 4_9_0; with 4_8_2 it passes. Do you have a standalone make command to check where it fails? I can't seem to find how to get more messages beside a returned -2 from gradle.

Also, building with 4_8_2 and running with 4_9_0 gets me another message of 'spawn failed', so we need same version for compile and run.

Constantin Asofiei wrote:

Done, but FWD native task fails when building with 4_9_0;

I've used the pre-built archive here.

Constantin Asofiei wrote:

Constantin Asofiei wrote:

Done, but FWD native task fails when building with 4_9_0;

I've used the pre-built archive here.

Use gradlew native --info to get more info for failure. The required *.h and *.lib files are not in MinGW binaries for libffi and jvm.lib.

Also, building with 4_8_2 and running with 4_9_0 gets me another message of 'spawn failed', so we need same version for compile and run.

I think we need to use 4.9.0 MinGW. It is known as working in VM.

Eugenie Lyzenko wrote:

Constantin Asofiei wrote:

Constantin Asofiei wrote:

Done, but FWD native task fails when building with 4_9_0;

I've used the pre-built archive here.

Use gradlew native --info to get more info for failure. The required *.h and *.lib files are not in MinGW binaries for libffi and jvm.lib.

Yup, I've missed jvm.lib, now build passed.

Another issue: jvm.dll must be on the PATH (and I think it might need to be compatible with the jvm.lib which was used to compile spawn).

After this, the web client gives the same The stub received bad data. error.

Constantin Asofiei wrote:

Another issue: jvm.dll must be on the PATH (and I think it might need to be compatible with the jvm.lib which was used to compile spawn).

Yes, jvm.dll and jvm.lib must be from the same JDK package.

After this, the web client gives the same The stub received bad data. error.

OK. I'm setting up new system to compare.

I've just checked with the standalone command produced by server.cmd -t and web client works...

Constantin Asofiei wrote:

I've just checked with the standalone command produced by server.cmd -t and web client works...

What JDK you use, 64-bit or 32-bit?

Eugenie Lyzenko wrote:

Constantin Asofiei wrote:

I've just checked with the standalone command produced by server.cmd -t and web client works...

What JDK you use, 64-bit or 32-bit?

64-bit

Constantin Asofiei wrote:

Eugenie Lyzenko wrote:

Constantin Asofiei wrote:

I've just checked with the standalone command produced by server.cmd -t and web client works...

What JDK you use, 64-bit or 32-bit?

64-bit

OK. This looks a king if issue I worked previously with server.cmd. The command file introduces environment variables that consumes memory allocated for environment. I have cleaned almost all of them. But not all can be removed. If you take a look at server.cmd you will see that after main server command there is some processing. The rc, statcode we can not clean.

Eugenie Lyzenko wrote:

OK. This looks a king if issue I worked previously with server.cmd. The command file introduces environment variables that consumes memory allocated for environment. I have cleaned almost all of them. But not all can be removed. If you take a look at server.cmd you will see that after main server command there is some processing. The rc, statcode we can not clean.

Something interesting: I've added these lines just above the set rc:

ENDLOCAL & SET fserverCmd=%serverCmd%

:: run the P2J server
CMD.EXE /C %fserverCmd%

ERROR: API        = CreateProcessWithLogonW
       error code = -2147024809
       message    = The parameter is incorrect.

I'm out of ideas at this time, another approach would be to replace CMD.EXE /C %serverCmd% with the standalong java command (so that everything else in the script does not affect the launch command) and just do a divide&conquer on the rest of the code: remove a portion, run, and see if web client passes... if it passes, and with the removed portion fails, then the error is somewhere in that code.

Constantin Asofiei wrote:

Eugenie Lyzenko wrote:

OK. This looks a king if issue I worked previously with server.cmd. The command file introduces environment variables that consumes memory allocated for environment. I have cleaned almost all of them. But not all can be removed. If you take a look at server.cmd you will see that after main server command there is some processing. The rc, statcode we can not clean.

Something interesting: I've added these lines just above the set rc:
[...]
and now I get a different kind of error:
[...]

I'm out of ideas at this time, another approach would be to replace CMD.EXE /C %serverCmd% with the standalong java command (so that everything else in the script does not affect the launch command) and just do a divide&conquer on the rest of the code: remove a portion, run, and see if web client passes... if it passes, and with the removed portion fails, then the error is somewhere in that code.

Can you please test increasing Java stack size to 2048k? Adding -Xss2048k to the server launch.

OK. I have installed new Windows 7 64-bit Pro. And web client has been started without errors.

Eugenie, -Xss2048k doesn't help. But I found the root cause: setting the cpath like this:

set cpath=.;..\lib\application.jar;..\lib\p2j.jar;

for %%f in (..\lib\*.jar) do call :add_jar %%f

I don't know what the problem is with the original for loop code...

Constantin Asofiei wrote:

Eugenie, -Xss2048k doesn't help. But I found the root cause: setting the cpath like this:
[...]
instead of this:
[...]
solves the issue, server.cmd works correctly.

This is great!

I don't know what the problem is with the original for loop code...

I think this should be in our TODO list. Because I have the original CMD file working fine. This is something we need to understand.

Greg,

I've made some investigations over the weekend for current spawn.exe implementation working with Windows for automated mode.

1. The FWD server is starting under dedicated user.
2. The web client is able to run with different users(including one that starts the server)

1. Windows XP working fine using embedded task scheduler. Not working with service for users other than service owner(0xC0000142 error).
2. Windows 7 has more issues. The only case when everything is OK is the case the user make interactive logon, manually starts the server and then run the client. In this case it is possible to use client for different users.
3. If the server is starting within Windows 7 task scheduler it is possible to run client for user that owns the server process. Other users causes the java client application to end abnormally with 0xC0000142 error as if the server is running as service.

May be it is required to have server edition of Windows - one of my suggestion. I think we need to rewrite win_spawn.c code to be compatible with automated execution to fix this issue. For example to implement other way to pass the data to the client application instead of using environment variables(pipe or queue).

So I need to know what is our plans here? Should I perform more investigations/fixes for C Windows code? Or the current priority is 1521 task?

Stop working on this. All future work will happen in #3264. We will defer that work until a later time.

Greg Shah wrote:

Stop working on this. All future work will happen in #3264. We will defer that work until a later time.

OK.