Feature #11742
implement a fully featured Debug Adapter Protocol (DAP) server for 4GL code (in FWD)
0%
Related issues
History
#1 Updated by Greg Shah 3 months ago
- Subject changed from implement a fully featured Debug Adapter Protocol server for 4GL code (in FWD) to implement a fully featured Debug Adapter Protocol (DAP) server for 4GL code (in FWD)
The Debug Adapter Protocol is a platform independent protocol for separating the debugging back-end from the debugger UI. As such it allows the same back-end debugging support to be used by multiple IDEs.
It is an open standard, originally created by Microsoft for the VSCode product. It is being adopted by multiple IDEs now and it is a kind of companion to the Language Server Protocol.
It is required to be built for our VSCode extension in #11355.
#2 Updated by Greg Shah 3 months ago
- Related to Feature #11355: VSCode extension added
#3 Updated by Greg Shah 3 months ago
- Related to Feature #11741: implement a fully featured Language Server Protocol (LSP) server for 4GL code added
#4 Updated by Paula Păstrăguș 22 days ago
I have some questions regarding this one:
1. Which source do we debug: the original 4gl file or the preprocessed cache file?
2. Do we debug only the 4GL, or should the converted Java also be visible? For example, a mixed call stack, or a "show Java" link to the companion view
3. May the conversion be changed to produce a 4GL line to Java line map? Today the generated Java carries no line info. If so, in which form, for example a .dbgmap would be a good fit?
4. Should the map be produced always, or only behind a setting in p2j.cfg.xml?
5. Should the debugger control the code from outside, attaching to the server's JVM like a Java debugger (JDI/JDWP)? That means no runtime changes and no cost when not debugging, but 4GL state has to be read out of FWD's Java objects. Or should it control it from inside, through hooks in the FWD runtime?
6. Attach to a running server only, or should the extension also launch the server and client and run a chosen program?
7. OpenEdge compatibility: should DEBUGGER:SET-BREAK and the other DEBUGGER handle features work?
8. What are your expectations for the very first phase of DAP?
#5 Updated by Greg Shah 21 days ago
1. Which source do we debug: the original 4gl file or the preprocessed cache file?
The cache file is the most obvious mapping. It makes the most sense to me. As long as we provide mechanisms (keyboard shortcuts and popup menu items) to navigate to the original sources (we need the ability to jump to the corresponding location in the editable file from which the code came).
2. Do we debug only the 4GL, or should the converted Java also be visible? For example, a mixed call stack, or a "show Java" link to the companion view
Optimally, we would be able to switch back and forth between either the 4GL or the Java debug views. The actual debugging should be a common session. The views just show the correct language and provide the UI for interaction.
3. May the conversion be changed to produce a 4GL line to Java line map? Today the generated Java carries no line info. If so, in which form, for example a .dbgmap would be a good fit?
Yes, we will need something like this. It needs to be bidirectional.
4. Should the map be produced always, or only behind a setting in p2j.cfg.xml?
It should always be produced. I think it can go into the same ./cvt/ directory structure with our other intermediate artifacts.
5. Should the debugger control the code from outside, attaching to the server's JVM like a Java debugger (JDI/JDWP)?
Yes.
That means no runtime changes and no cost when not debugging, but 4GL state has to be read out of FWD's Java objects.
Yes
Or should it control it from inside, through hooks in the FWD runtime?
No
6. Attach to a running server only, or should the extension also launch the server and client and run a chosen program?
As noted in Phase 1 IDE Support, we need the ability to launch servers, agents and clients etc.
7. OpenEdge compatibility: should DEBUGGER:SET-BREAK and the other DEBUGGER handle features work?
Evaluate the effort for this and create a task with some details about the idea and approach. I don't think we need this feature in this task, we will defer it.
8. What are your expectations for the very first phase of DAP?
Proper step-level debugging of 4GL source code with full variable support. It is OK if this must connect to an already running server/clients. Jumping back and forth between Java and 4GL is also pretty important.
#6 Updated by Paula Păstrăguș 21 days ago
- Related to Feature #11985: implement the DEBUGGER system handle on top of the 4GL DAP server added
#7 Updated by Paula Păstrăguș 6 days ago
- Status changed from New to WIP
FWD 4GL debugger (DAP) and VS Code extension: what works today (2026-10-29)
Overview¶
The FWD 4GL debugger debugs converted 4GL programs from VS Code at the level of 4GL statements. It attaches to a running FWD server and works on the preprocessed .cache file of each program in the conversion folder (cvt/…/<program>.p.cache, .w.cache, .cls.cache): breakpoints, stack frames and steps all refer to .cache lines, include files included. A toggle switches the same session to the generated Java at any time, and the Java View shows the generated Java beside its .cache file, kept in sync by the cursor and by debug stops. The FWD runtime is unchanged: the server only needs JDWP enabled.
Getting started¶
- Convert the project with this FWD build and run
ant deploy, so every program gets a<program>.dbgmapfile next to its.cache. - Build FWD in the project's
p2jtree (./gradlew jar); the extension starts the debug adapter from there. The adapter needs a JDK (settingfwd4gl.server.java, defaultjava). - In the extension folder (
tools/vscode-4gl):npm install(first time only),npm run compile, then F5 "Run Extension (clean)", and open the converted project folder (the one withcfg/p2j.cfg.xml) in the new window. - Start the server with JDWP:
cd deploy/server && ./server.sh -d(JDWP port 2080). The server then runs interpreted, so everything is slower. - Add the attach configuration to
launch.json(also offered as the snippet "FWD 4GL: Attach to FWD server"):
{ "type": "fwd-4gl", "request": "attach", "name": "Attach to FWD server",
"host": "localhost", "port": 2080 }
6. Optional attribute projectRoot points to the converted project when it is not the open folder. Start the debug session, open cvt/…/<program>.cache, set breakpoints, then run the client.
Same FWD build on both sides. The adapter and the server must run the same FWD build. Otherwise the Debug Console shows FWD version mismatch and Buffers, Shared, System, handle attributes and buffer fields show <unavailable: FWD version mismatch> and Set Value is refused (breakpoints, stepping, stack, Locals and Program still work). Fix: rebuild FWD, run ant deploy.prepare (or ant deploy) in the project, restart ./server.sh -d and attach again.
Attach errors say what is wrong, e.g. No JDWP agent at host:port (start the server with -d) or JDWP refused the connection; another debugger may be attached (only one debugger can attach to a server).
Debugging features¶
Breakpoints¶
- Set on lines of
.cachefiles. A breakpoint on a blank, comment,DEFINEorENDline moves to the next statement with code in the same block; the dot shows the adjusted line. - A breakpoint that cannot be placed stays unverified with the reason (
no 4GL map,map stale: …,class not loaded yet— installed when the class loads). - Breakpoints work in loops (every iteration), in several programs at once, on include lines, and can be added or removed while stopped.
- Conditions (Edit Breakpoint → Expression): a 4GL expression; stops only when it is
yes. An error or a non-LOGICAL result stops and printsBreakpoint condition error: …. - Hit counts, counted over hits whose condition held; they restart when any breakpoint of the file is edited:
| Form | Stops at |
|---|---|
N or ==N |
the Nth hit only |
>N, >=N, <N, <=N |
hits compared with N |
%N |
every Nth hit |
- Logpoints (Log Message): each
{expression}is replaced by its value (or<error: …>), the line goes to the Debug Console and the thread runs on.
Stepping and threads¶
- Step Over, Step Into and Step Out move by 4GL statement and follow the real control flow (LEAVE, NEXT, UNDO/RETRY, ON ERROR, CATCH/FINALLY, RETURN).
- Step Into enters procedures, functions, methods, constructors, property accessors, triggers, external and persistent procedures (RUN, RUN … IN, DYNAMIC-FUNCTION, super procedures, PUBLISH subscribers, RUN VALUE).
- Step Over runs a called routine at full speed; Step Out returns to the next statement of the caller.
- DEFINE statements are not stop points; a step stops only once per line (
IF x THEN NEXT.stops once). - Continue and Pause act on the selected thread only.
- Each client session is a thread (
Session <id> (<account>); appserver workers as<Type> worker #<n>). Only the session that hits a breakpoint stops; the others run on. - Disconnect removes the breakpoints, resumes the stopped threads and leaves the server and all clients running.
Call stack¶
- In 4GL mode each 4GL routine is one frame on its
.cacheline, e.g.start.p:42 (FOR EACH)for the main block andprocess-value (start.p:95)for an internal procedure. FWD runtime frames are hidden; recursion and triggers get their own frames. - Selecting an outer frame shows its variables and evaluates expressions in it.
- A program without a usable map appears as a dimmed Java frame marked
(no 4GL map)or(map stale).
4GL ⇄ Java mode¶
Switch with the debug toolbar button "4GL ⇄ Java" (arrow-swap icon), the status bar item FWD: 4GL / FWD: Java, or the command "4GL ⇄ Java" in the command palette. Stopped threads stay where they are.
| 4GL mode | Java mode | |
|---|---|---|
| Threads | Sessions, workers, stopped threads | All JVM threads |
| Stack | 4GL frames on .cache lines |
Every Java frame on .java lines |
| Variables | 4GL scopes | this and Java locals |
| Stepping | By 4GL statement | By Java line |
| Evaluate | 4GL expressions | Not available |
Breakpoints stay on .cache lines in both modes.
Variables¶
| Scope | Contents |
|---|---|
| Locals | Parameters (with their mode, e.g. n (INPUT)) and variables of the routine; for the main block, the program variables; the variable of a CATCH block, only while its CATCH runs |
| Program | Program-level (class-level) variables, in internal procedures, functions and methods; STATIC data members with the value of the stopped session |
| Buffers | Buffers in scope, temp-tables (default buffer), BUFFER parameters |
| Shared | Shared and global shared variables, buffers, temp-tables |
| System | ERROR-STATUS, SESSION, THIS-PROCEDURE, SOURCE-PROCEDURE, TARGET-PROCEDURE, SELF |
- Values look like 4GL:
"text",42,yes/no,?, dates in the sessionDATE-FORMAT, DECIMAL asSTRING()shows it, RAW and MEMPTR as a hex dump. - Extents show
[n]and expand to their elements (pages of 100). - Handles show id and type, e.g.
1002 (PROCEDURE), and expand to their attributes. A handle the program never asked the id of shows only(PROCEDURE). An attribute whose getter reports a 4GL error shows<error: ** … (nnnn)>in place of the value; the debugged session is not affected (no message to the user,ERROR-STATUSunchanged, nothing reaches its CATCH). - Objects show their 4GL class name and expand to data members and properties, inherited ones included. STATIC members show the value of the stopped session, or
<not set in this session yet>before the session used the class. - Buffers show
record available, NO-LOCKorno recordand expand to the fields of the current record under their 4GL names. - Computed properties (GET with a body) are lazy: expanding runs the GET once.
THIS-PROCEDUREis the procedure of the selected frame.
Watch, hover, Debug Console¶
Available in 4GL mode on the selected frame. The expression language (also used by conditions and logpoints) has literals (numbers, strings, yes/no, ?), names, name[i], buffer.field, h:ATTR, obj:Member, the operators OR AND NOT = <> < > <= >= BEGINS + - * / (and EQ NE LT GT LE GE) with 4GL semantics, and parentheses. There are no function calls, assignments, date literals or system handles as names (SESSION:…).
- Watch and Debug Console results that are extents, handles, objects or buffers expand as in the Variables view.
- Hover shows the reference under the mouse (name,
buffer.field,name[i],h:ATTRchains); keywords, strings and comments give no hover. A hover never runs a computed property's GET; Watch and Debug Console do.
Set Value¶
"Set Value" in the Variables view takes an expression. Allowed: variables, parameters and extent elements of Locals and Program of type CHARACTER, LONGCHAR, INTEGER, INT64, DECIMAL, LOGICAL; ? sets the unknown value. A STATIC data member is set in the stopped session only. The value is stored as an ordinary 4GL assignment (undo applies). Refused, with a message: other types (DATE, DATETIME, HANDLE, objects, RAW, MEMPTR, RECID, ROWID), properties, buffers, buffer fields, object members, Shared and System, a whole extent, out-of-range numbers, and text into a number.
Java View¶
Shows the generated Java of a program beside its .cache and keeps both in step, with or without a debug session.
- Open: command "FWD: Open Java View",
Ctrl+Alt+J, or the title-bar button on a.cacheeditor (from a generated.javait opens the.cache). The two also link by themselves when the.cacheand its.javaare both visible in two editors. - Cursor sync, both ways: the cursor on a 4GL statement highlights all its Java lines, and a Java line highlights its 4GL statement. This includes DEFINE statements (their declaration), PROCEDURE/FUNCTION/method headers (method signature), routine
ENDlines (closing brace) and blockENDlines of DO/FOR EACH/REPEAT. - Debug stops: in 4GL mode the Java shows the statement's lines subtly and the exact line the JVM is at strongly; in Java mode the
.cacheshows the 4GL statement line. The highlight follows the selected Call Stack frame and clears on Continue or a step. - The view follows the program of the active editor and never takes the focus.
- The Java tab carries a green
Jbadge while the map is current and a yellowSwhen the.cacheor.javachanged since conversion; the status bar then showsFWD Java View: map stale — reconvertand no highlights are drawn. - The Java file of the pair is read-only while linked.
- Closing the Java tab ends the link; hiding it behind another tab or moving it to another group does not.
Limitations¶
- Attach only: no launch of server or clients from VS Code.
- Breakpoints only in
.cachefiles, not in the.p/.w/.cls/.isources or in.javafiles. - No exception, function or data breakpoints; no
DEBUGGERsystem handle (#11985). - Expressions: no function calls, assignments, date literals or system handles as names.
- Set Value: only the types and targets listed above.
- Step Over a RUN that reads many records is about twice as slow as without the debugger.
DO j = 1 TO nrevisits its header line at every iteration (REPEAT/FOR EACHdo not).SOURCE-PROCEDUREandTARGET-PROCEDUREare shown for the innermost frame only.- A handle attribute whose getter reports a 4GL error (e.g. DATA-SOURCE-COMPLETE-MAP of a buffer outside a dataset) shows
<error: ** … (nnnn)>in place of the value; the debugged session is not affected. The OpenEdge debugger shows?there. - The variable of a CATCH block is shown only while its CATCH runs; a built-in error object (
Progress.Lang.SysError,AppError) does not show its properties yet, and methods such aserr:GetMessage(1)cannot be called from a watch. - Watches run computed-property GETs again at every stop.
- A thread stopped by Pause shows no Buffers, Shared, System or handle attributes; step once to inspect it.
- A value read that takes more than 5 s shows
<timed out>; that thread reads no more values until its next stop. - Computed EXTENT properties show their backing field.
- Java View:
DEFINE FRAME, widgets (BUTTON, RECTANGLE, IMAGE, BROWSE) andMENUhave no Java lines; triggerEND, property GET/SETENDand a few otherENDlines are not mapped. - Hover reads one line, so a line inside a multi-line comment gives no hover.
- This extension cannot run together with the #11741 extension yet: both have the same extension id.
Useful tips¶
- After a commit or rebuild of FWD:
./gradlew jar,ant deploy.preparein the project (ant deploywhen 4GL code or conversion changed), restart./server.sh -d, attach again; skipping this gives the version mismatch. - After changing a 4GL program, reconvert and redeploy it; until then its breakpoints stay unverified with
map staleand the Java View shows theSbadge. - After an update of the extension:
npm run compileand reload the extension window. - Output channel FWD Java View gives the details of Java View failures (link failures, unreadable maps, missing Java lines).
- Debug Console messages explain refused actions and stale maps.
Status¶
- All code (debug adapter,
.dbgmapgeneration in conversion, VS Code extensiontools/vscode-4gl) is committed in a local git repository (branch 11742a, head15dc430). Nothing is committed to bazaar yet. - Remaining: include the changes in the bazaar branch 11741a, together with the LSP, as several themed commits (being prepared); merge
tools/vscode-4glwith the #11741 extension (onlyextension.ts,package.jsonandREADME.mdneed a manual merge; the debugger files add without clashes).
#8 Updated by Greg Shah 5 days ago
I need to review this further but I wanted to get some initial feedback to you.
1. This is EXCELLENT! I love it. Great work!
2. We can't operate purely on a "single step per line" model. Something like IF x THEN NEXT. is 2 steps because it is 2 statements. Otherwise you can't evaluate x without also executing NEXT. That is not OK.
3. Everything in the debugger should be symmetrical. By this I mean that if a feature works in the 4GL, it must also work in Java and switching back and forth should be natural. For example, if a 4GL variable is in the watchlist, then the Java version of it is in the Java watchlist.
#10 Updated by Paula Păstrăguș 5 days ago
Thank you, Greg!
I will update my previous note (note 7), since several improvements were made after it was written.
About your points:
- One step per statement, not per line. Agreed. Today a step stops at most once per line, so
IF x THEN NEXT.evaluatesxand executesNEXTin a single step. I will change stepping to stop at every 4GL statement, also when several statements share a line, and to show which statement of the line is the current one, soxcan be inspected beforeNEXTruns. - Symmetry between 4GL and Java. Agreed. Every feature which works in 4GL mode should work in Java mode too, and switching should carry the context over: a 4GL variable in the watch list shows as its Java counterpart in Java mode, and back. Today Java mode shows the Java stack and locals and steps by Java line, but has no expression evaluation (so no watches, hovers or conditions), and breakpoints can only be set on
.cachelines.
I will try to have as much as possible done by tomorrow, and I will post the result here.
#11 Updated by Greg Shah 5 days ago
All of this needs to be in the same branch as the LSP and all the ANTLR4 changes. Assuming 11741a was branched from 10804a, then I guess combining the DAP and LSP in 11741a will work fine.
Post the proposed package structure here for review and discussion. I want a clean design that puts the LSP, DAP and VS Code extension into its own directory tree.
#12 Updated by Paula Păstrăguș 5 days ago
11742a was not branched from 10804b, neither 11741a. The LSP still lacks auto-complete support since it relies on the ANTLR4 grammar to track valid next tokens after a 'partial' statement. I was planning to defer this past October 30 since 10804b is still awaiting review, and while I was working on the ANTLR deep dive doc, the AI surfaced several issues (which was documented) that we need to resolve first.
#13 Updated by Paula Păstrăguș 5 days ago
Proposed structure: LSP, DAP and the VS Code extension
The language server and the debug adapter each already have their own package tree, and the extension has its own directory. I propose to keep that layout and to bring both into one branch (even branched from 10804b, if needed):
src/com/goldencode/p2j/
lsp/ the language server (46 classes), LspLauncher is its main class
kb/ knowledge base resources
dap/ the debug adapter (38 classes), DapLauncher is its main class
eval/ the 4GL expressions of watch, hover, conditions, logpoints and set value
vars/ scopes and variable values, read from the stopped server over JDI
target/ SilentGetter, the one class which runs inside the FWD server
dbgmap/ the .dbgmap model, its JSON reader and writer, and a command-line check
tools/vscode-4gl/ the one VS Code extension, which starts both servers
- The conversion writes the
.dbgmapfiles:convert/DebugMapWorker.java, hooked inrules/convert/brew.xmlandcontrol_flow.rules, with the line counting inio/ByteCountOutputStreamandpattern/CommonAstSupport. It usesdbgmap, which the debug adapter reads. - The front end serves the language server: in
preproc/the source provider, the supplied includes and the origin hints (#3881); inuast/the check-only mode ofAstGeneratorand the annotationsprogress.gadds in a check-only parse. - The FWD server loads
dap/target/SilentGetterby name, so it ships inp2j.jarlike the rest.
Merging the two branches
- The Java packages do not overlap:
lspcomes from 11741a,dapanddbgmapfrom 11742a (which is currently a git repo) - The extension is merged into one, with one id and one
package.json:- From 11742a: the debugger and the Java View (
debug.ts,dbgmap.ts,expressions.ts,javaView*.ts,javaIndex.ts,javaText.ts,javaBadge.ts). - From 11741a: everything else, including the grammar.
extension.ts,package.jsonandREADME.mdare the only files that exists in both, 11742 and 11741.
- From 11742a: the debugger and the Java View (
#15 Updated by Paula Păstrăguș 5 days ago
Right. I also updated #11742-7 note with the latest improvements.
#18 Updated by Greg Shah 5 days ago
Please move #11742-7 into a wiki page. Then edits are versioned and we can refer to it later without reading a task.
Individual commits should have a post here with good details. That way we can follow the work in the task but always find the latest docs in the wiki page.
#19 Updated by Paula Păstrăguș 5 days ago
Would it make sense to have a single wiki page in the Internal Documentation covering both 11741 and 11742, since they are related?