Feature #3759
setup, test and document PostgreSQL on Windows
100%
History
#1 Updated by Eric Faulhaber over 5 years ago
- Assignee set to Ovidiu Maxiniuc
#2 Updated by Ovidiu Maxiniuc over 5 years ago
A few questions here:
- Will the database server run on same machine as the FWD server?
Theoretically, this should not matter because we are connecting usinghost:port
in connection string fromdirectory.xml
. The only difference would be the latency imposed by networking. - Is there a preferred PostgreSQL version I should focus on? I am running currently on 9.2 / 9.4 but the latest version is 11.1 (which I understand is not yet available on Ubuntu platforms).
I understand that the result will be integrated in our wiki similar to https://proj.goldencode.com/projects/p2j/wiki/Database_Server_Setup_for_PostgreSQL_on_Linux or https://proj.goldencode.com/projects/p2j/wiki/Database_Server_Setup_for_SQLServer_on_Windows.
#3 Updated by Greg Shah over 5 years ago
- Will the database server run on same machine as the FWD server?
Yes.
- Is there a preferred PostgreSQL version I should focus on? I am running currently on 9.2 / 9.4 but the latest version is 11.1 (which I understand is not yet available on Ubuntu platforms).
Use the same version that is the latest current version for 16.04 (9.5?) since that is the latest version that we will have tested. If there is some issue with that version on Windows, then you can drop back to an older supported version that will avoid the issue.
I'd don't want to mix up moving to a new version with the Windows change.
I understand that the result will be integrated in our wiki similar
Correct.
#4 Updated by Ovidiu Maxiniuc over 5 years ago
- Status changed from New to WIP
I am having some difficulties with building PL/Java on Windows. Apparently, msvc profile is selected by Maven. I do not understand why. Of course it will fail because there is no 'link' in MinGW. If I manually select the 'compiler-mingw64' profile I get a minute of scrolling errors but the root cause seems to be int64 support.
#5 Updated by Greg Shah over 5 years ago
Eugenie: Please review this task and offer any advice about mingw.
#6 Updated by Ovidiu Maxiniuc over 5 years ago
- File mingw.profile.log
added
- File default.profile.log
added
Please find the attached logs from
mvn clean install > default.profile.log
and
mvn clean install -P compiler-mingw64 > mingw.profile.log
I noticed that the msvc
profile has an additional include path: ${PGSQL_INCLUDEDIR-SERVER}/port/win32_msvc
. As a last resort, I tried to add it to mingw profile in the hope that there is some support for int64 there, but it does not have any effect.
The second question is: why mvn uses the msvc
as default build profile? The answer might be in
<activation>
<property>
<name>env.VCINSTALLDIR</name>
</property>
</activation>
as opposed to
<activation>
<property>
<name>env.MSYSTEM</name>
<value>MINGW64</value>
</property>
</activation>
...but I could not find any of such system variables (I might be wrong here, this is my first encounter with POM structure at this level).
#7 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
Please find the attached logs from
[...]
and
[...]I noticed that the
msvc
profile has an additional include path:${PGSQL_INCLUDEDIR-SERVER}/port/win32_msvc
. As a last resort, I tried to add it to mingw profile in the hope that there is some support for int64 there, but it does not have any effect.The second question is: why mvn uses the
msvc
as default build profile? The answer might be in
[...]
as opposed to[...]
...but I could not find any of such system variables (I might be wrong here, this is my first encounter with POM structure at this level).
Try to add system variable:MSYSTEM
to be MINGW64
What version of the Maven
do you use?
#8 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Try to add system variable:
MSYSTEM
to beMINGW64
I will try that.
What version of the
Maven
do you use?
Apache Maven 3.6.0 and gcc (GCC) 8.2.0.
#9 Updated by Ovidiu Maxiniuc over 5 years ago
After I
set MSYSTEM=MINGW64
the mingw profile is indeed used. However, the issues remain.
#10 Updated by Eugenie Lyzenko over 5 years ago
- File aol.properties.mingw64 added
Ovidiu Maxiniuc wrote:
After I
[...]
the mingw profile is indeed used. However, the issues remain.
Take a look at your .m2\repository\com\git-hub\maven-nar\nar-maven-plugin\version
directory. The version
is 3.2.3
in my system, you can have another number. There is a file nar-maven-plugin-version.jar
inside directory.
This is module used to build native code and is failing now. Take inside the jar(easy in Ubuntu), go to the com/git-hub/maven-nar
tree inside jar. You will see the aol.properties
file. Extract it temporary and review, may be you have to patch this config file.
The aol.properties
has a set of the compiler/linker references to instruct maven
what to use with compile/link and options for compile/link. If the file has wrong values - the native module will not be built. I have attached the sample file for you to compare. I guess the lines in file are pretty clear to see the meaning.
If you will patch the file - the name(aol.properties
) and the location inside jar must be exactly the same.
#11 Updated by Eugenie Lyzenko over 5 years ago
Also checklist for PLJava:
https://tada.github.io/pljava/build/build.html
Nar plugin:
http://maven-nar.github.io/
#12 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Take a look at your
.m2\repository\com\git-hub\maven-nar\nar-maven-plugin\version
directory. Theversion
is3.2.3
in my system, you can have another number. There is a filenar-maven-plugin-version.jar
inside directory.
I have this jar: .m2\repository\com\github\maven-nar\nar-maven-plugin\3.2.3\nar-maven-plugin-3.2.3.jar
(note the path is a bit different, but this is not relevant).
If you will patch the file - the name(
aol.properties
) and the location inside jar must be exactly the same.
I did that. The log did changed a bit, but the problems are still there.
I tracked the source of the error message (#error must have a working 64-bit integer datatype
) and I got to c:\Program Files\PostgreSQL\9.5\include\server\c.h
. At lines 328-356 I can read (I added indentation so that it is more visible):
/*
* 64-bit integers
*/
#ifdef HAVE_LONG_INT_64
/* Plain "long int" fits, use it */
#ifndef HAVE_INT64
typedef long int int64;
#endif
#ifndef HAVE_UINT64
typedef unsigned long int uint64;
#endif
#define INT64CONST(x) (x##L)
#define UINT64CONST(x) (x##UL)
#elif defined(HAVE_LONG_LONG_INT_64)
/* We have working support for "long long int", use that */
#ifndef HAVE_INT64
typedef long long int int64;
#endif
#ifndef HAVE_UINT64
typedef unsigned long long int uint64;
#endif
#define INT64CONST(x) (x##LL)
#define UINT64CONST(x) (x##ULL)
#else
/* neither HAVE_LONG_INT_64 nor HAVE_LONG_LONG_INT_64 */
#error must have a working 64-bit integer datatype
#endif
I did some digging and HAVE_LONG_LONG_INT_64
macro that is conditionally defined in c:\Program Files\PostgreSQL\9.5\include\server\pg_config.h
like this:
/* Define to 1 if `long long int' works and is 64 bits. */
#if (_MSC_VER > 1200)
#define HAVE_LONG_LONG_INT_64 1
#endif
IIRC, MSC_VER
is defined as 1200 in Visual Studio 6.0
. And most likely not defined at all in g++.
The another macro HAVE_LONG_INT_64
is used as a fallback, but I could not find its definition anywhere.
#13 Updated by Eugenie Lyzenko over 5 years ago
As far as I know the MinGW64 has support for long long int
. it is __int64
. So what you need to do is to try to define:-DHAVE_LONG_LONG_INT_64
and -Dint64=__int64'
for CPPFLAGS
related entries in aol.properties
you are going to use.
My understanding is in MinGW
the unsigned __int64
is a UINT64
.
#14 Updated by Ovidiu Maxiniuc over 5 years ago
I finally get it 100% compiled. The problem is that psql does not know what are the 'resources' (read 'header declarations') of the compiler comes with. The .h files check some flags and if not set then psql API adds its own declarations. I finally used the following line in aol.properties
:
amd64.Windows.gpp.c.defines=Windows HAVE_LONG_LONG_INT_64 HAVE_GETTIMEOFDAY HAVE_STRUCT_SOCKADDR_STORAGE HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY PG_PRINTF_ATTRIBUTE=gnu_printf
#15 Updated by Ovidiu Maxiniuc over 5 years ago
I think I was a little to enthusiastic about the successful build. Apparently there is an issue with the binaries because the extension cannot be created. After further investigations I found these lines in a log file:
LOG: server process (PID 6028) was terminated by exception 0xC0000005 DETAIL: Failed process was running: create extension pljava HINT: See C include file "ntstatus.h" for a description of the hexadecimal value. LOG: terminating any other active server processes WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. LOG: all server processes terminated; reinitializing
I could not find any ntstatus.h
anywhere on my HDD, but the error code 0xC0000005 seems to be quite known. A look in MSDN (https://msdn.microsoft.com/en-us/library/cc704588.aspx) tells that it's STATUS_ACCESS_VIOLATION. This looks bad because I don't think we can what happens behind the scene, when pljava-so-1.5.0.dll
is loaded and what is wrong with it.
#16 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
I think I was a little to enthusiastic about the successful build. Apparently there is an issue with the binaries because the extension cannot be created. After further investigations I found these lines in a log file:
[...]I could not find any
ntstatus.h
anywhere on my HDD, but the error code 0xC0000005 seems to be quite known. A look in MSDN (https://msdn.microsoft.com/en-us/library/cc704588.aspx) tells that it's STATUS_ACCESS_VIOLATION. This looks bad because I don't think we can what happens behind the scene, whenpljava-so-1.5.0.dll
is loaded and what is wrong with it.
Yes, this is the real issue. Please check:
1. If the DLL binary bitness matches the OS and other DB components that is working with. I mean bitness of the rest DB binaries.
2. Did you set up MinGW
compile/link flags properly. Comparing to P2J.DLL
makefile options.
3. Whether the pljava-so-1.5.0.dll
uses mixed MSVC
and MinGW
codes together. Especially on the DLL
init stage.
4. May be it makes sense to look into C native code to find out some code that can be incompatible with MinGW
or currently used OS. For example some Windows subsystem can be really 32-bit only even with 64-bit of the OS.
Also it would be great to have more debugging info for 0xC0000005
. The module name and address will help significantly. What the Windows Error logger tells for exception details?
#17 Updated by Eric Faulhaber over 5 years ago
https://tada.github.io/pljava/build/mingw64.html implies that the PostgreSQL server needs to have been built using mingw, in order to build PL/Java with mingw. Is that the case here?
#18 Updated by Ovidiu Maxiniuc over 5 years ago
Also it would be great to have more debugging info for 0xC0000005. The module name and address will help significantly. What the Windows Error logger tells for exception details?
I don't have the memory address that caused the exception. The Windows' Event Viewer reports only the messages I wrote above.
#19 Updated by Ovidiu Maxiniuc over 5 years ago
Eric Faulhaber wrote:
https://tada.github.io/pljava/build/mingw64.html implies that the PostgreSQL server needs to have been built using mingw, in order to build PL/Java with mingw. Is that the case here?
Yes. Compiling with mingw is optional I think, but this is the way we do it - as opposed to using msvc complier/linker.
#20 Updated by Ovidiu Maxiniuc over 5 years ago
I tried today with a fresh new system, hoping that the initial system could have been somehow corrupted. There is no change. I encountered the exact behaviour.
Eugenie Lyzenko wrote:
Yes, this is the real issue. Please check:
1. If the DLL binary bitness matches the OS and other DB components that is working with. I mean bitness of the rest DB binaries.
I inspected the native built library, PLJAVA-SO-1.5.0.DLL. It only depends on KERNEL32.DLL, MSVCRT.DLL and POSTGRES.EXE. All of them are 64-bit binaries.
2. Did you set up
MinGW
compile/link flags properly. Comparing toP2J.DLL
makefile options.
P2J.DLL depends on a few more DLL files. All are 64-bit. I don't recall using this file. I only used the server side of FWD when working on Windows.
3. Whether the
pljava-so-1.5.0.dll
uses mixedMSVC
andMinGW
codes together. Especially on theDLL
init stage.
I cannot tell that, but which MSVC
code? I do not have any installed. Except for the standard redistributable that were installed automatically. Note that the machine has SQL Server 2012 up & running.
4. May be it makes sense to look into C native code to find out some code that can be incompatible with
MinGW
or currently used OS. For example some Windows subsystem can be really 32-bit only even with 64-bit of the OS.
based on my dependency investigations I think we can exclude any 32 bit issues.
I re-read some articles on web and found this:
If your PostgreSQL server was built using MinGW-w64 (which you can check by running pg_config and looking for mingw), then you can build PL/Java using the same toolchain.
This is strange. As I understand this now, this is a possible cause why PL/Java compiled with MinGW? Apparently my PostgreSQL is not built with MinGW. I do not know what was used for building PostgreSQL I downloaded and installed.
However, I tried to find such a binary, of PostgreSQL built with MinGW. I could only find something like PostgreSQL 9.3 but I really do not trust the sources.
#21 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
I tried today with a fresh new system, hoping that the initial system could have been somehow corrupted. There is no change. I encountered the exact behaviour.
Eugenie Lyzenko wrote:
Yes, this is the real issue. Please check:
1. If the DLL binary bitness matches the OS and other DB components that is working with. I mean bitness of the rest DB binaries.I inspected the native built library, PLJAVA-SO-1.5.0.DLL. It only depends on KERNEL32.DLL, MSVCRT.DLL and POSTGRES.EXE. All of them are 64-bit binaries.
2. Did you set up
MinGW
compile/link flags properly. Comparing toP2J.DLL
makefile options.P2J.DLL depends on a few more DLL files. All are 64-bit. I don't recall using this file. I only used the server side of FWD when working on Windows.
Did you use MinGW
options -Wl,--subsystem,windows,--kill-at
while building PL/Java module?
#22 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
If your PostgreSQL server was built using MinGW-w64 (which you can check by running pg_config and looking for mingw), then you can build PL/Java using the same toolchain.
This is strange. As I understand this now, this is a possible cause why PL/Java compiled with MinGW? Apparently my PostgreSQL is not built with MinGW. I do not know what was used for building PostgreSQL I downloaded and installed.
However, I tried to find such a binary, of PostgreSQL built with MinGW. I could only find something like PostgreSQL 9.3 but I really do not trust the sources.
Do you mean this build: https://sourceforge.net/projects/postgresql-mingw-w64/ ?
#23 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Do you mean this build: https://sourceforge.net/projects/postgresql-mingw-w64/ ?
I downloaded the package at location. There are two archives inside - for 32 and 64 bit. But they only contain some plugin libraries:libcurl-4.dll
, libiconv-2.dll
, liblber.dll
, libldap.dll
, libmysqld.dll
, libsqlite3-0.dll
, libxml2-2.dll
, libyajl.dll
, oci.dll
with their support bundles (*.control
, *.sql
, etc).
So this is not ProgreSQL server.
#24 Updated by Ovidiu Maxiniuc over 5 years ago
I found an installed a PostgreSQL that is apparently built with MinGW. It is available here: https://www.openscg.com/bigsql/postgresql/installers.jsp/, see "PostgreSQL 9.5.15" frame. It installs a bit different, in a different location and with a bit different directory structure. The pg_config
reports the following:
BINDIR = C:/Apps/POSTGR~1/pg95/bin DOCDIR = C:/Apps/POSTGR~1/pg95/share/doc/POSTGR~1 HTMLDIR = C:/Apps/POSTGR~1/pg95/share/doc/POSTGR~1 INCLUDEDIR = C:/Apps/POSTGR~1/pg95/include PKGINCLUDEDIR = C:/Apps/POSTGR~1/pg95/include/POSTGR~1 INCLUDEDIR-SERVER = C:/Apps/POSTGR~1/pg95/include/POSTGR~1/server LIBDIR = C:/Apps/POSTGR~1/pg95/lib PKGLIBDIR = C:/Apps/POSTGR~1/pg95/lib/POSTGR~1 LOCALEDIR = C:/Apps/POSTGR~1/pg95/share/locale MANDIR = C:/Apps/PostgreSQL/pg95/share/man SHAREDIR = C:/Apps/POSTGR~1/pg95/share/POSTGR~1 SYSCONFDIR = C:/Apps/PostgreSQL/pg95/etc/postgresql PGXS = C:/Apps/POSTGR~1/pg95/lib/POSTGR~1/pgxs/src/MAKEFI~1/pgxs.mk CONFIGURE = '--prefix=C:/msys64/opt/pgbin-build/pgbin/bin/../20181106_0039/build/pg95-9.5.15-1-win64' '--enable-integer-datetimes' '--enable-thread-safety' '--with-libxml' '--with-libxslt' '--with-ossp-uuid' '--enable-nls' '--with-openssl' '--with-ldap' '--with-python' '--with-perl' '--with-tcl' '--with-libraries=/opt/pgbin-build/pgbin/shared/win64/lib:/usr/local/lib:/usr/local/ssl/lib' '--with-includes=/usr/local/include:/usr/local/ssl/include' 'CFLAGS=-O2 -DMS_WIN64 -I/opt/pgbin-build/pgbin/shared/win64/include' CC = gcc CPPFLAGS = -I./src/include/port/win32 -DEXEC_BACKEND -I/usr/local/include/libxml2 -I/usr/local/include -I/usr/local/ssl/include -I../../../src/include/port/win32 CFLAGS = -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -DMS_WIN64 -I/opt/pgbin-build/pgbin/shared/win64/include CFLAGS_SL = LDFLAGS = -Wl,--allow-multiple-definition -Wl,--disable-auto-import -L/usr/local/lib -LC:/msys64/mingw64/lib -L/opt/pgbin-build/pgbin/shared/win64/lib -L/usr/local/lib -L/usr/local/ssl/lib -Wl,--as-needed LDFLAGS_EX = LDFLAGS_SL = LIBS = -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssl -lcrypto -lz -lws2_32 -lm -lws2_32 VERSION = PostgreSQL 9.5.15Notice
-LC:/msys64/mingw64/lib
in LDFLAGS
. There are some problems here:
- the paths do not match the paths from my system. For example, the -L directive above should have read
-LC:/Apps/msys64/mingw64/lib
. - some paths are UNIX-like.
I tried to adjust the paths for a closer match and started maven. It stops with the following error:
[ERROR] C:\Apps\POSTGR~1\pg95\include\POSTGR~1\server/c.h:99:10: fatal error: libintl.h: No such file or directory [ERROR] #include <libintl.h> [ERROR] ^~~~~~~~~~~ [ERROR] from C:\Apps\pljava-master\pljava-so\src\main\include/pljava/PgObject.h:12, [ERROR] compilation terminated.
In fact it repeats itself for different files (20-30 times).
I do not have such file libintl.h
anywhere on my drive. Apparently, it is missing from PostgreSQL's include or it was available when it was built. I think this PostgreSQL distribution was cross-compiled.
#25 Updated by Greg Shah over 5 years ago
I suspect you will need to build the project yourself from source. In this way you can ensure that everything is matching.
#26 Updated by Greg Shah over 5 years ago
Ovidiu: Are you close to resolving this? If not, I propose that Eugenie can take the next steps. He is quite a wizard with native code issues. :)
#27 Updated by Ovidiu Maxiniuc over 5 years ago
Greg Shah wrote:
Ovidiu: Are you close to resolving this? If not, I propose that Eugenie can take the next steps. He is quite a wizard with native code issues. :)
I agree. If Eugenie can solve the build/install of PL/Java, it would be great. I do not have many ideas. I was about to attempt to compile PL/Java with MSVC compiler. Just to see it works. There was the free Visual Studio Express
, but now they offer Visual Studio Community
(https://visualstudio.microsoft.com/free-developer-offers/) which is advertised as free.
Anyway, I an thinking whether we should offer the PL/Java package already pre-build (I believe we do similar for libffi
) because of these issues.
After that, I will continue updating the https://proj.goldencode.com/projects/p2j/wiki/Database_Server_Setup_for_PostgreSQL_on_Windows with the rest of instructions. It is completed up to Load FWD's User Defined Function Library
. The ending part is not in final form.
#28 Updated by Greg Shah over 5 years ago
I was about to attempt to compile PL/Java with MSVC compiler. Just to see it works. There was the free Visual Studio Express, but now they offer Visual Studio Community (https://visualstudio.microsoft.com/free-developer-offers/) which is advertised as free.
I think this is a good idea. Eugenie is unavailable until the end of the week, so please do try this compile to see if it resolves the issue.
If it doesn't work, then Eugenie can take this when he gets back, unless you have other ideas that can be checked quickly.
Anyway, I an thinking whether we should offer the PL/Java package already pre-build (I believe we do similar for libffi) because of these issues.
Agreed.
#29 Updated by Ovidiu Maxiniuc over 5 years ago
I installed Visual Studio 2017 Community edition. I cannot build. Maven keeps complaining that Could not launch cmd.exe /X /C "gcc --version"
.
Of course, I 'deactivated' gcc by altering its path so the OS will not find it. Otherwise Maven will build with success.
I tried to force Maven to use the newly installed msvc:- dropped the
MSYSTEM
environment variable; - added
VCINSTALLDIR=c:\Progra~2\MIB055~1\2017\Community\VC
(see note-6 above for the reason of these two); - added the path to
cl
compiler andlink
linker (it looks likeC:\PROGRA~2\MIB055~1\2017\Community\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64
on my system) to environment's PATH variable; - force msvc build profile in Maven command line:
mvn clean install -P compiler-msvc
; - cleaned Maven cache (by removing
.m2
directory from home dir). - altered
aol.properties
to update/add following lines:x86.Windows.msvc.linker.options=/LIBPATH:C:\\PROGRA~2\\WI3CF2~1\\10\\Lib\\10.0.17763.0\\ucrt\\x86 /LIBPATH:C:\\PROGRA~2\\WI3CF2~1\\10\\Lib\\10.0.17763.0\\um\\x86 /LIBPATH:C:\\PROGRA~2\\MIB055~1\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x86 x86.Windows.msvc.c.options=-IC:\\PROGRA~2\\MIB055~1\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\include -IC:\\PROGRA~2\\WI3CF2~1\\10\\Include\\10.0.17763.0\\ucrt -IC:\\PROGRA~2\\WI3CF2~1\\10\\Include\\10.0.17763.0\\um -IC:\\PROGRA~2\\WI3CF2~1\\10\\Include\\10.0.17763.0\\shared amd64.Windows.msvc.linker.options=/LIBPATH:C:\\PROGRA~2\\WI3CF2~1\\10\\Lib\\10.0.17763.0\\ucrt\\x64 /LIBPATH:C:\\PROGRA~2\\WI3CF2~1\\10\\Lib\\10.0.17763.0\\um\\x64 /LIBPATH:C:\\PROGRA~2\\MIB055~1\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64 amd64.Windows.msvc.c.options=-IC:\\PROGRA~2\\MIB055~1\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\include -IC:\\PROGRA~2\\WI3CF2~1\\10\\Include\\10.0.17763.0\\ucrt -IC:\\PROGRA~2\\WI3CF2~1\\10\\Include\\10.0.17763.0\\um -IC:\\PROGRA~2\\WI3CF2~1\\10\\Include\\10.0.17763.0\\shared
Eventually, the build was successful:
[INFO] Reactor Summary for PostgreSQL PL/Java 1.5.0: [INFO] [INFO] PostgreSQL PL/Java ................................. SUCCESS [ 1.203 s] [INFO] PL/Java API ........................................ SUCCESS [ 2.125 s] [INFO] PL/Java backend Java code .......................... SUCCESS [ 2.438 s] [INFO] PL/Java backend native code ........................ SUCCESS [ 41.041 s] [INFO] PL/Java Deploy ..................................... SUCCESS [ 6.250 s] [INFO] PL/Java Ant tasks .................................. SUCCESS [ 13.468 s] [INFO] PL/Java examples ................................... SUCCESS [ 1.422 s] [INFO] PL/Java packaging .................................. SUCCESS [ 3.375 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS
Generating
pljava-1_5_0-msvc\pljava-packaging\target\pljava-pg9.5-amd64-Windows-msvc.jar
.
Unfortunately, after the extension is installed, the database server is reset again. I can see this in pg_log
:
2018-12-03 23:20:53 EET LOG: database system is ready to accept connections 2018-12-03 23:23:33 EET LOG: server process (PID 2052) was terminated by exception 0xC0000005 2018-12-03 23:23:33 EET DETAIL: Failed process was running: CREATE EXTENSION pljava 2018-12-03 23:23:33 EET HINT: See C include file "ntstatus.h" for a description of the hexadecimal value. 2018-12-03 23:23:33 EET LOG: terminating any other active server processes 2018-12-03 23:23:33 EET WARNING: terminating connection because of crash of another server process 2018-12-03 23:23:33 EET DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2018-12-03 23:23:33 EET HINT: In a moment you should be able to reconnect to the database and repeat your command. 2018-12-03 23:23:33 EET WARNING: terminating connection because of crash of another server process 2018-12-03 23:23:33 EET DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2018-12-03 23:23:33 EET HINT: In a moment you should be able to reconnect to the database and repeat your command. 2018-12-03 23:23:33 EET LOG: all server processes terminated; reinitializing
#30 Updated by Greg Shah over 5 years ago
- Assignee changed from Ovidiu Maxiniuc to Eugenie Lyzenko
OK, thanks for the update.
Eugenie will take this when he gets back.
#31 Updated by Eugenie Lyzenko over 5 years ago
My started config environment:
1. PostgreSQL 9.5.15 downloaded as binary (to be on the almost same level as in Linux)
2. GCC version 4.9.0 (mingw64)
3. Oracle JDK 1.8.0_192
4. PLJava 1.5.2 source zip
The aol.properties
with changes described in note #3759-14. The building fails with errors below(failure on linking stage).
Ovidiu,
Have you encountered this issue with your build? Do you have MSYS
installed package? The command mvn clean install
was executed from MSYS
shell or from regular command prompt?
Native compilation issue log:
... [ERROR] C:\opt\pljava_152\pljava-so\src\main\c\SubXactListener.c: In function 'subXactCB': [WARNING] C:\opt\pljava_152\pljava-so\src\main\c\SubXactListener.c:26:2: warning: enumeration value 'SUBXACT_EVENT_PRE_COMMIT_SUB' not handled in switch [-Wswitch] [ERROR] switch(event) [ERROR] ^ [ERROR] C:\opt\pljava_152\pljava-so\src\main\c\XactListener.c: In function 'xactCB': [WARNING] C:\opt\pljava_152\pljava-so\src\main\c\XactListener.c:25:2: warning: enumeration value 'XACT_EVENT_PARALLEL_COMMIT' not handled in switch [-Wswitch] [ERROR] switch(event) [ERROR] ^ [WARNING] C:\opt\pljava_152\pljava-so\src\main\c\XactListener.c:25:2: warning: enumeration value 'XACT_EVENT_PARALLEL_ABORT' not handled in switch [-Wswitch] [WARNING] C:\opt\pljava_152\pljava-so\src\main\c\XactListener.c:25:2: warning: enumeration value 'XACT_EVENT_PRE_COMMIT' not handled in switch [-Wswitch] [WARNING] C:\opt\pljava_152\pljava-so\src\main\c\XactListener.c:25:2: warning: enumeration value 'XACT_EVENT_PARALLEL_PRE_COMMIT' not handled in switch [-Wswitch] [WARNING] C:\opt\pljava_152\pljava-so\src\main\c\XactListener.c:25:2: warning: enumeration value 'XACT_EVENT_PRE_PREPARE' not handled in switch [-Wswitch] [INFO] 53 files were compiled. [INFO] 53 files were compiled. [INFO] Linking... [INFO] Linking... [INFO] Starting link {4.9.0 -Wl,--export-all-symbols -m64 -Wl,--subsystem,windows,--kill-at -shared -LC:\POSTGR~1\9.5\lib -Bdynamic -lpostgres -shared-libgcc} [INFO] Starting link {4.9.0 -Wl,--export-all-symbols -m64 -Wl,--subsystem,windows,--kill-at -shared -LC:\POSTGR~1\9.5\lib -Bdynamic -lpostgres -shared-libgcc} [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x131): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x13f): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x166): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x189): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x197): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x1b1): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x1bf): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x210): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x21e): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x245): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x268): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x276): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x290): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x29e): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x2ff): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x30d): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x338): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x445): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x453): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x46d): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x47b): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x4cd): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x4db): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x502): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x539): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x547): undefined reference to `__imp_error_context_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x561): undefined reference to `__imp_PG_exception_stack' [ERROR] C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o:AclId.c:(.text+0x56f): undefined reference to `__imp_error_context_stack' [ERROR] c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\opt\pljava_152\pljava-so\target\nar\obj\amd64-Windows-gpp\AclId.o: bad reloc address 0x0 in section `.data' [ERROR] collect2.exe: error: ld returned 1 exit status ...
#32 Updated by Eugenie Lyzenko over 5 years ago
- File libintl.h
added
- File aol.mingw.properties
added
Looks like I have worked PLJava extension. What was changed:
0. MSYSTEM
must be MINGW64
1. Upgrade MAVEN
to 3.6.0
2. Install PostgreSQL 9.5.15. This is the MinGW
based build from https://www.openscg.com/bigsql/postgresql/installers.jsp/.
3. Use aol.mingw.properties
provided here.
4. Install libintl.h
into MinGW64
include dir provided here(actually I guess this is just make compiler to be happy).
5. Install openssl
header file directory into MinGW64
include dir. Need to copy full directory from MSVC
based PostgreSQL
build. This is the serious issue of the MinGW
based build and I have no explanation why these files are missing there. This looks like "almost complete package" approach allowing to use PostgreSQL
as application but stopping to use it for development.
6. Run mvn clean install -Dnar.aolProperties=aol.mingw.properties
to build extension. Was OK on my system. The command must be run in regular shell, not in VC
environment.
7. Install extension by java -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
8. Final steps:
- psql
- SET pljava.libjvm_location TO 'C:\some\pathh\...\jvm.dll';
- CREATE EXTENSION pljava;
- GRANT USAGE ON LANGUAGE java TO PUBLIC;
- restart the server to make sure it works.
Ovidiu, please confirm if this scenario works on your system. Or please let me know what approach did you use to verify the new extension is working.
My conclusion is the main issue is PostgreSQL
and PLJava
must be build using the same compiler(MinGW in our case).
I have no PLJava build for MSVC
based PostgreSQL
so I still do not know why this combination does not work. May be we need to investigate this case additionally. Continue working here. During several PostgreSQL
installations(now I have 3 different DB servers running, 9.4, 9.5(MS), 9.5(MinGW)) the different MS runtimes was also installed so need to investigate if the build can work without them.
#33 Updated by Eugenie Lyzenko over 5 years ago
One addition. The system used is Windows 7 64-bit Ultimate
.
#34 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Looks like I have worked PLJava extension. What was changed:
0.MSYSTEM
must beMINGW64
1. UpgradeMAVEN
to 3.6.0
2. Install PostgreSQL 9.5.15. This is theMinGW
based build from https://www.openscg.com/bigsql/postgresql/installers.jsp/.
3. Useaol.mingw.properties
provided here.
I tried with all above. The aol.mingw.properties
extracts only the mingw
-related directives from packaged aol.properties
. The only difference is that I used a much newer gcc compiler:
gcc --version gcc (Rev3, Built by MSYS2 project) 8.2.0
I used both msys2 and standard console. Indeed, building from msys2 console was not successful.
4. Install
libintl.h
intoMinGW64
include dir provided here(actually I guess this is just make compiler to be happy).
I stopped here see end of note 24 above. Where did you get libintl.h
? It is legal to distribute the file alone? In fact this was the reason my build with PostgreSQL 9.5.15 (from https://www.openscg.com/bigsql/postgresql/installers.jsp/) was not successful.
5. Install
openssl
header file directory intoMinGW64
include dir. Need to copy full directory fromMSVC
basedPostgreSQL
build. This is the serious issue of theMinGW
based build and I have no explanation why these files are missing there. This looks like "almost complete package" approach allowing to usePostgreSQL
as application but stopping to use it for development.
I did not reach this point when building mingw
version of PSQL. My successful builds using mingw
and msvc
of MSVC
based PostgreSQL
did not require this.
6. Run
mvn clean install -Dnar.aolProperties=aol.mingw.properties
to build extension. Was OK on my system. The command must be run in regular shell, not inVC
environment.
7. Install extension byjava -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
This requires special rights. I used Administrator console for successful builds.
8. Final steps:
- psql
- SET pljava.libjvm_location TO 'C:\some\pathh\...\jvm.dll';
I was not aware of this setting in psql interface. I have this instead added to postgresql.conf
:
pljava.libjvm_location = 'C:\PROGRA~1\Java\JDK18~1.0_1\jre\bin\server\jvm.dll'
- CREATE EXTENSION pljava;
This is where my build binaries failed:
postgres=# CREATE EXTENSION pljava; server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The connection to the server was lost. Attempting reset: Failed. !>
- GRANT USAGE ON LANGUAGE java TO PUBLIC;
- restart the server to make sure it works.Ovidiu, please confirm if this scenario works on your system. Or please let me know what approach did you use to verify the new extension is working.
I need to cleanup and rebuild using your instructions. I will be back in about 1 h with the result.
My conclusion is the main issue is
PostgreSQL
andPLJava
must be build using the same compiler(MinGW in our case).
I also tried with msvc
compiler (both PostgreSQL
and PLJava
) and I got same exception. What is strange is that I get the same crash for 3 different PC (two at the office and my personal laptop at home).
I have no PLJava build for
MSVC
basedPostgreSQL
so I still do not know why this combination does not work. May be we need to investigate this case additionally. Continue working here. During severalPostgreSQL
installations(now I have 3 different DB servers running, 9.4, 9.5(MS), 9.5(MinGW)) the different MS runtimes was also installed so need to investigate if the build can work without them.
#35 Updated by Ovidiu Maxiniuc over 5 years ago
Yes, this time the extension was created with success. I also upgraded to 1.5.2 (I used 1.5.0 previously). I do not know if this made any difference.
The only issues I encountered were at step 5. Beside openssl (I misunderstood at first and downloaded and installed true openssl), the full server\port\win32\
directory must also be copied to include
directory of mingw. Most of these files only contains commented code but they are needed by compiler.
- that this mingw version of PostgerSQL does not come with the openssl directory
- why
win32
was not recognized. If I look intopq_config
output I can clearly see:INCLUDEDIR-SERVER = C:/Apps/POSTGR~1/pg95/include/POSTGR~1/server CPPFLAGS = -I./src/include/port/win32 -DEXEC_BACKEND -I/usr/local/include/libxml2 -I/usr/local/include -I/usr/local/ssl/include -I../../../src/include/port/win32
The path to PostgreSQL is correct, but the paths to desired includes seem broken. The actual absolute path is:
c:\Apps\PostgreSQL\pg95\include\postgresql\server\port\win32\
#36 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
...
3. Use
aol.mingw.properties
provided here.I tried with all above. The
aol.mingw.properties
extracts only themingw
-related directives from packagedaol.properties
.
This allows to use it in command line leaving original copy in maven jar untouched.
The only difference is that I used a much newer gcc compiler:
[...]
Why do you need the compiler other than we provide for FWD
package? The 4.9.0
is known as working but I can not tell the same for other versions(including newer one) because if I recall we had an issues with some of them(especially with Windows console client).
Do we have a plans to upgrade our prepackaged versions.
4. Install
libintl.h
intoMinGW64
include dir provided here(actually I guess this is just make compiler to be happy).I stopped here see end of note 24 above. Where did you get
libintl.h
? It is legal to distribute the file alone? In fact this was the reason my build with PostgreSQL 9.5.15 (from https://www.openscg.com/bigsql/postgresql/installers.jsp/) was not successful.
I guess its is legal: https://github.com/git-for-windows/git-sdk-64/blob/master/mingw64/include/libintl.h.
I have no PLJava build for
MSVC
basedPostgreSQL
so I still do not know why this combination does not work. May be we need to investigate this case additionally. Continue working here. During severalPostgreSQL
installations(now I have 3 different DB servers running, 9.4, 9.5(MS), 9.5(MinGW)) the different MS runtimes was also installed so need to investigate if the build can work without them.
The question remaining unclear for me - how can I get the MSVC
based PostgreSQL and Mingw64
based compiled PLJava combination?
#37 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Ovidiu Maxiniuc wrote:
This allows to use it in command line leaving original copy in maven jar untouched.
That's cool.
Why do you need the compiler other than we provide for
FWD
package? The4.9.0
is known as working but I can not tell the same for other versions(including newer one) because if I recall we had an issues with some of them(especially with Windows console client).
I didn't do it on purpose. I just installed msys2 and when updating, pacman
installed the newer gcc
and make
.
Do we have a plans to upgrade our prepackaged versions.
Yes, Greg agreed on this (see note 28, above).
I guess its is legal: https://github.com/git-for-windows/git-sdk-64/blob/master/mingw64/include/libintl.h.
OK.
The question remaining unclear for me - how can I get the
MSVC
based PostgreSQL andMingw64
based compiled PLJava combination?
I do not know yet. The interesting thing is that PL/Java built with MSVC
Community Edition is also failing to create the extension for me. Now, I have an idea that I would like to test: I have both PostgreSQL-s installed. Do you have any idea how can I alter
java -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
so that the extension go to MSVC instance?
I know that this is a long-shot, but I would give it a try.
#38 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
The question remaining unclear for me - how can I get the
MSVC
based PostgreSQL andMingw64
based compiled PLJava combination?I do not know yet. The interesting thing is that PL/Java built with
MSVC
Community Edition is also failing to create the extension for me. Now, I have an idea that I would like to test: I have both PostgreSQL-s installed. Do you have any idea how can I alterjava -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
so that the extension go to MSVC instance?
I think it is not possible via changing jar internals. I see only class that can be responsible for proper directory handling: org/gjt/cuspy/JarX.class
. No config files inside, so nothing to adjust to new path. However you can try to change PATH
variable in a way the pg_config
command to be executed from new location. This can help but I'm not sure.
#39 Updated by Eugenie Lyzenko over 5 years ago
Or as alternative you can try to manually copy the files from your postgresql
subdir to new location:
1. complete pljava
directory.
2. extension/pljava.control
file.
#40 Updated by Ovidiu Maxiniuc over 5 years ago
I have good news.
The pljava 1.5.2
package that I built today installs on both instances of PostgreSQL. Indeed, you need to alter the Windows's PATH environment variable so that target pg_config
to be find first. There is a catch: to install on msvc instance you need some Administrator rights while the mingw instances does not need any additional rights. This is because the msvc instance is installed in Program Files
system folder.
Why this? I do not know, I only can speculate that there is some kind of issue with pljava 1.5.0
.
#41 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
I have good news.
The
pljava 1.5.2
package that I built today installs on both instances of PostgreSQL. Indeed, you need to alter the Windows's PATH environment variable so that targetpg_config
to be find first. There is a catch: to install on msvc instance you need some Administrator rights while the mingw instances does not need any additional rights. This is because the msvc instance is installed inProgram Files
system folder.Why this? I do not know, I only can speculate that there is some kind of issue with
pljava 1.5.0
.
I have another theory. The functions __imp_PG_exception_stack
and __imp_error_context_stack
the MinGW
attempting to link with is missing in MSVC
PostgreSQL package. There are no such exports in postgres.lib
. The postgres.lib
library also uses them so the target is somewhere inside Visual Studio
libraries I guess. Even if GCC
linker ignores this missing calls resolution on building stage(as it happens in your new GCC
version) - it can just postpone the trouble to the execution stage. Just because the native code still needs the functions.
#42 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
...
Do we have a plans to upgrade our prepackaged versions.
Yes, Greg agreed on this (see note 28, above).
Sorry, I would have to be more clear. This was a question. Do we really have a plans to upgrade MinGW
prebuilt package we provide to configure developer workstation? I do not see any related in #3759-28 above.
This is serious point because in the case we want to upgrade we need to get complete MinGW
sources, recompile it in some Windows environment with MSYS
and package only MinGW
. As far as I know we do not provide MSYS
prebuilt binaries, so user needs only ready to use MinGW
package. Moreover we will need to retest all our native programs/modules that are built with MinGW
. And I think rebuild libffi
as well.
In general if all of us will use different versions of GCC
it makes debugging pretty problematic due to different results on different systems. Moreover newer version does not automatically mean the better one.
#43 Updated by Greg Shah over 5 years ago
I am open to changing our packaged version if needed to support all functionality. What do you recommend?
We need to make it reasonable to get things like this built. We can also publish our Windows binaries for PL/Java, right?
#44 Updated by Eugenie Lyzenko over 5 years ago
The addition. My previous note about MinGW
was wrong. We do not build MinGW
itself. We use already compiled binaries from SourceForge
. Currently the latest build is from 20180726
. I'm not sure what the GCC
version is.
#45 Updated by Eugenie Lyzenko over 5 years ago
Greg Shah wrote:
...
We need to make it reasonable to get things like this built. We can also publish our Windows binaries for PL/Java, right?
Yes, we can publish our Windows binaries for PL/Java
. We just need to create and test complete install instruction to replace java -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
command from PL/Java
site which I think will not work due to different PosgreSQL
directory installations on different systems.
#46 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Yes, we can publish our Windows binaries for
PL/Java
. We just need to create and test complete install instruction to replacejava -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
command fromPL/Java
site which I think will not work due to differentPostgreSQL
directory installations on different systems.
Actually this works. At least it worked for me. I build PL/Java using MinGW and MinGW-built PostgreSQL installed in C:\Apps\ProgreSQL\pg9.5\
and I installed it on MSVC-built PostgreSQL which was installed in standard location, C:\Program Files\PostgreSQL\9.5
.
As far as I can tell, the java -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
command will use the first accessible pg_config
from PATH and install at respective location. Since I temporarily renamed the PostgreSQL
used for building and added the bin
of the msvc-built PostgreSQL instance to PATH, the jar installed to the later location. I tested and the EXTENSION was installed with success.
#47 Updated by Eugenie Lyzenko over 5 years ago
Greg Shah wrote:
I am open to changing our packaged version if needed to support all functionality. What do you recommend?
To answer this question I need to know what is the version of GCC
the recent build provides. Then I would search the differences and new features the new version offers. Then if we find the new features are important for us to use or open some attractive perspectives - I would do very careful regression testing for all native programs/modules we build with GCC
. Then if everything is OK - we can merge to new GCC
version.
If new version is the same or has no attractive features for us or if there are stability issues with new version - my way will be to keep working with current 4.9.0
. This is the known approach "Do not fix anything that works"
.
#48 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
Eugenie Lyzenko wrote:
Yes, we can publish our Windows binaries for
PL/Java
. We just need to create and test complete install instruction to replacejava -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
command fromPL/Java
site which I think will not work due to differentPostgreSQL
directory installations on different systems.Actually this works. At least it worked for me. I build PL/Java using MinGW and MinGW-built PostgreSQL installed in
C:\Apps\ProgreSQL\pg9.5\
and I installed it on MSVC-built PostgreSQL which was installed in standard location,C:\Program Files\PostgreSQL\9.5
.As far as I can tell, the
java -jar pljava-packaging/target/pljava-pg9.5-amd64-Windows-gpp.jar
command will use the first accessiblepg_config
from PATH and install at respective location. Since I temporarily renamed thePostgreSQL
used for building and added thebin
of the msvc-built PostgreSQL instance to PATH, the jar installed to the later location. I tested and the EXTENSION was installed with success.
This is a great news! We do not care about manual file copy in user documentation. Just configure proper pg_config
location.
#49 Updated by Eugenie Lyzenko over 5 years ago
Well, the new version of GCC
from 20180726
is 8.2.0
. Need to inspect what the new features we have got in this release.
#50 Updated by Ovidiu Maxiniuc over 5 years ago
Greg, how do we handle the missing header files issue? Evidently, we cannot ask the customers to install msvc-built PostgreSQL just because of the missing files? Will we provide them ?
#51 Updated by Greg Shah over 5 years ago
Ovidiu Maxiniuc wrote:
Greg, how do we handle the missing header files issue? Evidently, we cannot ask the customers to install msvc-built PostgreSQL just because of the missing files? Will we provide them ?
We probably should provide them. I prefer for them to be built into our mingw "distribution". Is that feasible? Are all the missing files safe to include (from a licensing perspective)?
#52 Updated by Ovidiu Maxiniuc over 5 years ago
Greg Shah wrote:
Beside the libintl.h which Eugenie attached there are two other directories:Ovidiu Maxiniuc wrote:
Greg, how do we handle the missing header files issue? Evidently, we cannot ask the customers to install msvc-built PostgreSQL just because of the missing files? Will we provide them ?
We probably should provide them. I prefer for them to be built into our mingw "distribution". Is that feasible? Are all the missing files safe to include (from a licensing perspective)?
openssl
: 75 files, most of them have OpenSSL project headers. Some are generated, some do not have headers at all.win32
directory has about a dozen files and sub-directories. Neither have any headers. Most of them do not have any active code at all, the others just chain include a couple of other system headers. The total size of this is 1KiB.
I think we can add them to our mingw "distribution".
#53 Updated by Greg Shah over 5 years ago
If our existing distribution will work with just adding these files, I'm OK with that approach.
#54 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
win32
directory has about a dozen files and sub-directories. Neither have any headers. Most of them do not have any active code at all, the others just chain include a couple of other system headers. The total size of this is 1KiB.I think we can add them to our mingw "distribution".
Strange but I do not need this to be copied in MinGW(even for GCC 8.2.0
) to build PL/Java binary.
Your PostgreSQL
distribution does not have the SomeRootDrivedir\PostgreSQL\pg95\include\postgresql\server\port\win32\
?
#55 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Strange but I do not need this to be copied in MinGW(even for
GCC 8.2.0
) to build PL/Java binary.
YourPostgreSQL
distribution does not have theSomeRootDrivedir\PostgreSQL\pg95\include\postgresql\server\port\win32\
?
Indeed, it is strange. I have it but the compiler complained about every single file from that directory. Copying it to mingw include directory fixed all issues. See my note-35 above.
#56 Updated by Eugenie Lyzenko over 5 years ago
The my points about new GCC
version update is the following. There are a lot of changes between 4.9.0
to 8.2.0
. The biggest one on my guess is shifting from C89
to C11
language specification.
Regarding to our native code usage in Windows:
- many sanity checks have been added to avoid wrong memory access in string operations and memcpy.
- pointer arithmetics is also "sanitized". I think it will be not so easy to use integer value as an object pointers as we do in COM
ActiveX
module. I expect more warnings.
- the Thread Local Storage support has been added. This is what I wanted to have while implementing COM
support. Should speed up small allocations/releases.
- another checks to improve dangerous code identifications on compilation stage.
- improved console readability and messages matching the real issue.
So taking into account the percentage of the native code in whole FWD
for now my point is merging can be useful but not urgent. I do not see any magic feature we need right now(correct me if I'm wrong). We can plan the shifting to new version when we have 2-4
days to rebuild all other natives(including libffi
) and test for critical regressions the spawn.exe
and p2j.dll
including ChUI
console support.
#57 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
Eugenie Lyzenko wrote:
Strange but I do not need this to be copied in MinGW(even for
GCC 8.2.0
) to build PL/Java binary.
YourPostgreSQL
distribution does not have theSomeRootDrivedir\PostgreSQL\pg95\include\postgresql\server\port\win32\
?Indeed, it is strange. I have it but the compiler complained about every single file from that directory. Copying it to mingw include directory fixed all issues. See my note-35 above.
Can you try to move your PostgreSQL
to C:\PostgreSQL\pg95\....
and check if it works?
#58 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie Lyzenko wrote:
Can you try to move your
PostgreSQL
toC:\PostgreSQL\pg95\....
and check if it works?
This is funny, now it works. I removed the win32
directory, so it does not take the file from there. So we can drop this part. Only openssl and libintl.h remain.
#59 Updated by Greg Shah over 5 years ago
We don't need to update the gcc version. Let's just include the missing files and update our distribution download.
We'll need to confirm that the changes don't break the FWD build.
#60 Updated by Eugenie Lyzenko over 5 years ago
Greg Shah wrote:
We don't need to update the gcc version. Let's just include the missing files and update our distribution download.
We'll need to confirm that the changes don't break the FWD build.
Agreed.
#61 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
Eugenie Lyzenko wrote:
Can you try to move your
PostgreSQL
toC:\PostgreSQL\pg95\....
and check if it works?This is funny, now it works.
What to say... This PostgreSQL
is an example of how to do not the package to deploy.
#62 Updated by Ovidiu Maxiniuc over 5 years ago
- File openssl.zip added
Added openssl header files needed to built PL/Java with mingw-built PostgreSQL.
#63 Updated by Ovidiu Maxiniuc over 5 years ago
I started to build PL/Java on customer system but at the end I stumbled on following issue: the javah
is missing from JDK 1.8.0_144. According to Oracle, has been superseded by the -h
option added to javac
in JDK 8, is deprecated as of JDK and might be removed in a future JDK release (https://docs.oracle.com/javase/9/tools/javah.htm#JSWOR687) but it seems that they seem to already drop it in latest updates of Java8 as I found already installed on the customer's system.
The replacement is the support from javac to do the very same job when launched with -h
parameter. However, the tools that build the project - nar
maven plugin does not support this. I read that they will add support 'for Java 10' in nar-maven-plugin-3.7.0, but when this will be released is unknown, there is only about a month from the release of 3.6.0 (latest release at this time).
#64 Updated by Greg Shah over 5 years ago
It is OK to install a pre-built (on your system) PL/Java on the customer system.
#65 Updated by Eugenie Lyzenko over 5 years ago
Several conclusions about building PL/Java
and modified MinGW
include directory with Windows environment:
1. The added libintl.h
and openssl
header file directory into MinGW
do not stop the FWD
project native modules compilation. It is safe to use.
2. The building PL/Java module was verified on clean system with only MinGW
based PostgreSQL
. So we can now say we have repeatable OK result. The absolutely mandatory setup points are to have proper MSYSTEM
as MINGW64
and respective custom aol.properties
file to use in maven command. No other extra tools(like MSYS
) are required.
3. When installing PostgreSQL
build the default directory(for program files) should be accepted to avoid include files finding issues.
4. Currently the 64-bit Windows 7 was tested. However I do not expect significant issues in further 64-bit Window versions.
5. 32-bit Windows case was not tested.
#66 Updated by Ovidiu Maxiniuc over 5 years ago
Eugenie,
I guess you refer in note-65 of MinGW-built PostgreSQL. Building with MSVC-built PostgreSQL is much simpler (it is the default profile and has all required include files) but the resulted binary fails at runtime.
Also I can confirm that the procedure works fine for systems running Windows10 and the equivalent server OS-es.
#67 Updated by Eugenie Lyzenko over 5 years ago
Ovidiu Maxiniuc wrote:
Eugenie,
I guess you refer in note-65 of MinGW-built PostgreSQL. Building with MSVC-built PostgreSQL is much simpler (it is the default profile and has all required include files) but the resulted binary fails at runtime.
Yes, I refer to the MinGW-built PostgreSQL. As the only way to get working binaries.
#68 Updated by Greg Shah over 5 years ago
What updates need to be made to the documentation chapters so that it is complete?
https://proj.goldencode.com/projects/p2j/wiki/Patching_and_Building_Customized_3rd_Party_Libraries#PLJava
https://proj.goldencode.com/projects/p2j/wiki/Database_Server_Setup_for_PostgreSQL_on_Windows
#69 Updated by Ovidiu Maxiniuc over 5 years ago
Greg Shah wrote:
What updates need to be made to the documentation chapters so that it is complete?
https://proj.goldencode.com/projects/p2j/wiki/Patching_and_Building_Customized_3rd_Party_Libraries#PLJava
https://proj.goldencode.com/projects/p2j/wiki/Database_Server_Setup_for_PostgreSQL_on_Windows
I updated Database Server Setup for PostgreSQL on Windows. But I linked the libintl.h
, aol.mingw.properties
and openssl.zip
to the attachment of this thread. I do not know if they are visible from there as this is a private tracker. I think they should be copied in a global location and linked from there.
#70 Updated by Greg Shah over 5 years ago
This thread is public, but I agree that the artifacts should be seen from the other two documents as well.
#71 Updated by Greg Shah over 4 years ago
- Status changed from WIP to Closed
- % Done changed from 0 to 100