Today we have faced strange problem. The customer was complainig, that his java stuff in the database is not working. The rror he has received was:
After some investigation, I found, that the database has crashed in the morning. The crash was caused by some ORA-600 error:
Few minutes after the crash one of my colleagues hast started the database. Then some new ORA-07745 occurred:
At this moment customer started to complain, that his java based procedures are failing with the division by zero error. I started to look on Metalink. In the ORA-600 and ORA-07745 Tool I have found one SR, in which someone else was facing same problem. One of the reasons could be, that the database was started with wrong LD_LIBRARY_PATH parameter. As this was for a solaris machine, the pldd command should be used to see, which libraries the process is using. So I run the command over the smon process:
And there was the problem. The 10.2.0.3 database was using 10.2.0.4 libraries. Therefore the division by zero and ORA-07745 errors. After this we have stopped the database, set correct LD_LIBRARY_PATH parameter and started the database. The ORA-07745 and java error were gone. But unfortunatelly another java error occured:
We have tried recompile whole java schema and also some other schemas in the database, but without success. We also opened a SR on metalink. On the other day, customer received some hints from the vendor of the application. They should run some udateStructure script. So we have restarted the database and customer run his script. After this, the problem was resolved. The only problem we should look after was, why there was wrong LD_LIBRARY_PATH, when we wer using oraenv script to change the variables. After few minutes it was clear, that the oraenv script is not setting correct LD_LIBRARY_PATH parameter. After logging on the server, the environment for the 10.2.0.4 was set as default. When you run oraenv script, the oraenv from 10.2.0.4 was started. And in this script there was no LD_LIBRAY_PATH parameter, so the default for 10.2.0.4 was kept. So that's why the database was started with wrong parameter. We have modified the script and now it is working without problems.