Deadlocks Concept 2
Deadlocks Concept 2
Deadlocks Concept 2
Tc.
When I checked ORACLE's v$lock, v$session, etc. when this problem occurred,
tcserver.exe kept the Engineering group record in ppom_object locked for about 2
hours.
If the above record remains DB locked for a long time, the user belonging to the
Engineering group cannot log in to Tc with error ORA-30006
(PPOM_OBJECT:RhCh8vpiRnTm7D).
Solution
1) Avoid running CLEARLOCKS utility during production time / working hours ( POM
does this automatically)
If any cron job or scheduled task is set as maintenance activity, remove CLEARLOCKS
utility from that cron job or scheduled task
2) Turn off External transaction ( completely turned off from tc13.1 onward) with
setting the variable: TC_DISABLE_TRANSACTIONS=1.
Historically, in Teamcenter the only way to remove locks held by dead sessions was
to run clearlocks [-verbose]. Starting Teamcenter 11.x, this behavior has changed.
POM has a lock challenger that removes any locks held by dead sessions
automatically.
Since POM automatically clears any locks held by dead sessions, Teamcenter does not
recommend customers to execute clearlocks during production hours.
Similarly the clear timestamp functionality can cause lock contention issues if run
in production time. To prevent this starting Teamcenter 11.x onwards clear
timestamp functionality has been modified to clear old timestamps.
-----------------------------------------------------------------------------------
-----------------------------
From last two days users are frequently facing the below issue:
"A database action has failed. The database may have timed out waiting for another
user's lock to be released or the database may be unavailable due to a hardware or
network failure. Please try your action again later. If this error continues,
please contact your system administrator."
Impact of this error is at the start some users sessions freezes while performing
any action like copy paste.
Then after sometime some users will not be able to login to 4 T RAC.
What can be the possible reason and solution for this. ?
http://gtac/view.php?sort=desc&p=1&q=A+database+action+has+failed.
+The+database+may+have+timed+out+waiting+for+another+user
%27s+lock+to+be+released&file_type=text&i=ir-9511412&k=9&o=0
<http://gtac/view.php?sort=desc&p=1&q=A+database+action+has+failed.
+The+database+may+have+timed+out+waiting+for+another+user
%27s+lock+to+be+released&file_type=text&i=ir-9511412&k=9&o=0 rel=noopener
noreferrer target=_blank>
Answer:
Please check if clear_locks, list_users utilities are run by any means of custom
code, scheduled jobs , etc during production working hours?
Perform any activity on the DB like reindexing, or any other activity?
Stop any background cron job or scheduled job or running maintenance activity /
utility like clear_locks, list_users during production hours.
Customer does not want to kill all processes on the server with clearlocks -
assert_all_dead.
What is the correct command line syntax to clearlocks for single a machine?
Solution
Please use synatx: clearlocks -assert_dead -u= -p= -g= <NodeName>
===================================================================================
==================================================
Oracle RAC is used to avoid the downtime during Windows Server maintenance
activity, however we are facing the issue for already connected Teamcenter
Session. Customer expecting that Teamcenter session should not be affected when
switching the oracle Node. Kindly clarify the Teamcenter behavior for ORACLE RAC
nodes.
clearlock
Search HistoryBasic SearchAdvanced Search
Translate to:
Select language
| Toggle to the original sourceA- | A | A+
Did you find your answer?
Yes
No
Don't know
FAQ
Questions regarding switching of Oracle RAC Nodes
Symptom
Customer is using Oracle12.2 RAC for Production environment.
We would like to know when switching ORACLE RAC (between Node1 & Node2), already
connected Teamcenter Session also be switched or need to restart the session?
Oracle RAC is used to avoid the downtime during Windows Server maintenance
activity, however we are facing the issue for already connected Teamcenter
Session. Customer expecting that Teamcenter session should not be affected when
switching the oracle Node. Kindly clarify the Teamcenter behavior for ORACLE RAC
nodes.
Does Teamcenter session need to restart after switching the nodes in RAC
configuration?
Please recommend some operation which will resend request to Oracle to confirm the
"fail over"?
Solution
The Teamcenter sessions should "fail over" to the node that remains up. But there
are qualifications:
The session loses its Oracle connection when the original node goes down, so it
only tries to reconnect when there is some Oracle request thus recommend:
- Don't run clearlocks or list_users when waiting for sessions to fail-over.
Until they try to reconnect they will be listed as not-connected and thus tidied
up. If the locks are used in another session then the original session will be
marked read-only upon reconnect.
- Sqlnet expire_time=10 (per Tdoc on Oracle config) is set so that there's a
keep-alive on the connection and we'll realize it's gone.
Do you mean to say that if user not making any operation (in Teamcenter Client)
which will send request to Oracle, the session will be tied up?
>> The TC server has the Oracle connection. If the connection is dropped (on fail-
over) the reconnect only happens if the TC client makes a request of the TC server
that requires a database call. We get an error from the DB client (Oracle OCI) to
say the connection is lost, and that prompts doing a reconnect. If the connection
disappears and the TC server for a given user isn't active then it'll show up as
not connected & risk of tidy-up via clearlocks.
Can you please explain the scenario for this case how lock will be used
in another session and it ended up with read-only upon reconnect? We would like to
understand this scenario to avoid this situation.
When attempting to patch from Teamcenter 10.1.1.1 to 10.1.6, TEM hangs at the
step where the update is running the clearlocks utility. There are no errors
reported in the UI and TEM does not quit/exit/or report errors in log files.
Solution
Upon investigation of the upgrade logs, it was discovered that multiple node names
no longer in use had processes that clearlocks was trying to kill.
Furthermore, running
clearlocks -verbose
from a tc cmd prompt also hung and did not complete. When testing a variety of
command line utilities, we found that those hitting the database also hung.
ie: list_users.
Using SQL to query how many process locks existed, it was discovered there were
over 250,000 locks. The majority of them were pointing to the node name no longer
in use.
Upon allowing TEM to run to completion (completed the clearlocks utility), the same
SQL query showed these locks cleared. Upon completion of the patch, Teamcenter
functioned properly.
===================================================================================
==============================================================================
Set Password:
=============================================================================
Clearlocks is trying to tidy old timestamps and it found around 8.7 million
rows to be deleted
Hence it is taking huge amount of time.
SELECT puid FROM POM_TIMESTAMP WHERE pdbtimestamp < (systimestamp AT TIME ZONE
'GMT' - 96/24);
INFO - 2018/09/16-02:43:21.422 UTC - NoId - ===>Took 214.623 seconds to
execute that SQL (returning 8735657 rows)