[solved]: CPU maxing out & won't publish

RLJB

This is a tip for anyone else (and a reminder for myself in 3 years time when I face this problem again)

We encountered an issue this week:

server getting slow to do certain things
publishing taking a VERY long time
restart of server and LONG Aware start-up time
publishing taking so long, it fails, CPU grows to 200% (even after adjusting memory, and having plenty of GB on server), throws Java Heap Space excceeded error or GC limit error

Solution:

2 DB tables had grown unusually large:

DV_RES - was 2.9GB
EXECUTION_CONTEXT - was 236MB

take a backup of 2 tables (just in case)
export all BSVs on the server (you will use them below)
shut down Aware
empty the contents of both tables (ie delete all rows)
turn back on Aware and publish all business spaces (if you don't do this you may be Visual Perspective issues when logging in)

Problem solved... can publish again, goes into test very quickly, restart Aware again in normal times

Powerm

Sounds great but it can happen after few weeks/months no ?

RLJB

Not sure Powerm.

We had been running this prod server for about 19 months. This is the first time we did this clean out.

Also FYI, after deleting DV_RES contents, and then reimporting the bsv's the DV_RES table grew back to about 250MB, not small, but no where near as big as 2.9GB.

Powerm

Thanks, good to know.

BenHayat

It would be great for Aware Team, to add a feature to Control Panel for us to click a button and clean those data, rather than growing through these steps. I think it should be an automated process.
Hope it goes on the "To-DO" list. 🙂

tford

Solution:

2 DB tables had grown unusually large:

DV_RES - was 2.9GB
EXECUTION_CONTEXT - was 236MB

take a backup of 2 tables (just in case)

export all BSVs on the server (you will use them below)

shut down Aware

empty the contents of both tables (ie delete all rows)

turn back on Aware and publish all business spaces (if you don't do this you may be Visual Perspective issues when logging in)

Problem solved... can publish again, goes into test very quickly, restart Aware again in normal times

Rod -- your solution was a life saver for me! I was experiencing java heap space errors and out of memory errors when trying to upload a BSV to a new version. I could not even delete a BSV in the config tool. After thinking I had memory issues or java issues on my server, I found your steps & they quickly solved my issue.

A HUGE thank you for posting the details here!

customaware

I had same issue a couple of weeks ago when I created
a process that got itself into an infinite loop. Kept pumping
stuff into EXECUTION CONTEXT with same result as you described
Boyd.
Actually had Vlad being kind enough to hunt it down the
day before the Conference in Vegas.

Just deleted the entire contents of EXECUTION CONTEXT and
all was good.

Apologies, should have posted it earlier.

tford

Just deleted the entire contents of EXECUTION CONTEXT and all was good.

I also got confirmation from Vladimir last night: "You can clear this table [EXECUTION CONTEXT] periodically when no one is working with the system"

himanshu

It really saves my life today.

Thanks RLJB and Mark.