Hello everyone. I have not been blogging for quite a while due to time pressure but it seems that now I’ve forced myself to start doing it again and honestly, more often than it was before. Six months ago I have been involved in Fusion Applications (next generation of Oracle Applications) POC internal setup in our company premises. That made me learn and investigate a lot of new things and products including Identity and Access Management, which is now playing a huge role in Fusion Applications setup (or provisioning – the correct word and replacement for installation when it comes to Fusion Apps). In fact, it is not even possible to install Fusion Apps itself, before the IdM part if fully and correctly configured. That’s why I have introduced two new categories on my blog, IdM and FA respectively. This year I had also presented the topic of IdM importance in Fusion Apps on UKOUG 2012 conference in Birmingham. So if someone is interested in that area, feel free to contact me and request the presentation slides or ask some questions on the subject.
Enough for the introduction, let’s get going. For those who are unfamiliar with what the heck is Webgate or IdM, I would suggest to read through some Oracle documentation first. For those who work with IdM and are using it for Single Sign-On, user management, etc., that application should be well-known. Anyway, here is an excerpt from official docs: “A WebGate is a Web server plug-in that is shipped out-of-the-box with Oracle Access Manager. The WebGate intercepts HTTP requests from users for Web resources and forwards them to the Access Server for authentication and authorization. An AccessGate is an Oracle Access Manager access client that processes requests for Web and non-Web resources and is developed using the Software Developer Kit“.
How it works? The Webgate is associated with an Oracle Access Manager (or Access Server) using Webgate Agent. OAM contains a list of protected and unprotected URL’s and other options used for access and interaction in your environment. In other words, you can define which resources (HTTP requests, for example), should be routed to OAM for authentication and authorisation before they could be accessed.
When it comes to Fusion Applications and IdM – those applications are closely integrated and IdM is used as a foot-stone for FA in access, user and role management, provisioning, etc. If you would need to install Fusion Applications in your environment, then you need to start with a mandatory part which is called Enterprise Deployment Guide for Oracle Identity Management Fusion Applications edition (or shortened EDG for FA). Be careful that it should be specifically for FA as it has a lot of additions there strictly for FA and also mind the version (11.1.5 the latest as of now) because there might be some new changes published based on the version of FA.
The issue I have encountered is there in Fusion Apps 11.1.4 (and I doubt it is fixed in 11.1.5 – I will explain later why is it so) and Webgate 11g. In previous versions Webgate 11g was not supported and 10g was there (in fact, 10g could still be used with 11.1.4 but why to go with so old versions). I had configured IdM part as per the documentation and faced no issues. Furthermore, Fusion Applications provisioning on top of the brand new IdM deployment went also fine, without any errors (it should error out a lot, if you have problems or did something wrong during setup of IdM). How big was my surprise that after the provisioning was done – I could not access most of the pages when Webgate/IdM was enabled it just simply gave the HTTP 404 error lead by The requested URL /oam/server/obrareq.cgi was not found! The even bigger surprise came when I have disabled the Webgate and was able to access the Fusion Applications home page successfully. But as I’ve described above, IdM plays a vital role here, so disabling the Webgate actually means to disable your whole environment from working properly.
I’ve started investigating it with a quick googling around and double checking the documentation which did not give any results at all. Everything seemed to be setup correctly. Enabling the detailed trace of OAM produced the following error when trying to access resources:
[2012-12-11T16:27:48.907+02:00] [wls_oam1] [NOTIFICATION] [OAM-02064] [oracle.oam.controller] [tid: [ACTIVE].ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)’] [userId: <anonymous>] [ecid: f8d11bad14c7f3a2:-26ec38c6:13aabfa3c8c:-8000-0000000000000011,1:136051] [APP: oam_server] Authentication Error during policy matching.
However, there shouldn’t be any problems with policies themselves, because Fusion Apps installer automatically populates and configures all policies into IdM during the provisioning phase. Then I found the following MOS note HTTP 404 Error Accessing A Protected Resource on an OHS Server Acting as a WLS Proxy in front of OAM [ID 1367723.1] which lead me to look for the issue in right direction. As per the note, by default a 11g WebGate denies access for non protected resources and the solution is to allow anonymous access to for URLs starting with /oam.:
Start the oamconsole and perform the following tasks:
1. Create the resources “/oam” and “/oam/…/*”
2. Add the resources to your public Authentication Policy, which uses the anonymous authentication schema.
3. Add the resources to your public Authorization Policy granting access to everyone.
Even though it looked suspicious because most probably that should’ve been added to EDG due to the fact that Fusion Apps is a protected resource of IdM by default in such integration. Making these changes did not resolve the issue, but made me pay more attention and investigate the statement that Webgate 11g denies access to all non-protected resources. Now let’s take a look into Webgate 11g Agent in oamconsole:
Mind the Deny On Non Protected attribute which is by default enabled and in fact, cannot be disabled from GUI. If you click on it, nothing just happens – it is simple as that. I don’t know what is the reason behind this “feature” and why is it designed so in latest version of Webgate, but I can ensure you that this was not there in Webgate 10g.
From this point, we have a clue that this might have caused such malfunction as HTTP 404 error. If we look into EDG for FA, then you would find a lot of different locations related to IdM, such as /oim (Oracle Identity Manager), /odsm (Oracle Directory Services Manager), etc. – all are vital in IdM and Fusion Apps administration. By default, those resources are non-protected and actually they should not be, because of the fact, that these resources are related to IdM itself. But even adding those resources to protected ones would not resolve the issue. So I thought of some other way to disable Deny On Non Protected flag. By default, the OAM configuration for FA is stored in oam-config.xml file located in the
<DOMAIN_HOME>/config/fmwconfig directory. Let’s have a look on what do we have there:
[oracle@idm fmwconfig]$ pwd /u01/app/oracle/admin/IDMDomain/mserver/IDMDomain/config/fmwconfig [oracle@idm fmwconfig]$ ls -l oam-config.xml -rw-r----- 1 oracle oinstall 338720 Jan 28 19:26 oam-config.xml
What we need is a Webgate 11g Agent we are using for integration with OAM, so let’s search for it:
</pre> <Setting Name="Webgate11g_Test_agent" Type="htf:map"> <Setting Name="AllowManagementOperations" Type="xsd:boolean">false</Setting> <Setting Name="state" Type="xsd:string">Enabled</Setting> <Setting Name="security" Type="xsd:string">open</Setting> <Setting Name="debug" Type="xsd:string">false</Setting> <Setting Name="SecondaryServerList" Type="htf:list"> </Setting> <Setting Name="failoverThreshold" Type="xsd:string">1</Setting> <Setting Name="version" Type="xsd:string">126.96.36.199</Setting> <Setting Name="id" Type="xsd:string">Webgate11g_Test_Agent</Setting> <Setting Name="denyOnNotProtected" Type="xsd:string">1</Setting> <Setting Name="logoutCallbackUrl" Type="xsd:string">/oam_logout_success</Setting> <Setting Name="logoutRedirectUrl" Type="xsd:string">http://my.company.com:7777/oam/server/logout</Setting> <Setting Name="cacheControlHeader" Type="xsd:string">no-cache</Setting> <Setting Name="PrimaryServerList" Type="htf:list"> <Setting Name="0" Type="htf:map"> <Setting Name="host" Type="xsd:string">my.company.com</Setting> <Setting Name="port" Type="xsd:string">5575</Setting> <Setting Name="numOfConnections" Type="xsd:string">1</Setting> </Setting> </Setting> <pre>
Notice the denyOnNotProtected Setting which is currently set to 1 (enabled). As you might already guessed, by changing it to 0, the setting would be disabled. Yes, I do admit it is simple, however you will not find it documented anywhere. For those, who are not very familiar with IdM, it might be tricky. The next steps would be to restart OAM and all other managed servers and also HTTP server where the Webgate plugin is applied. To make sure that the setting we just changed is in effect now monitor the file ObAccessClient.xml under your
<<OHS_INSTANCE>/config/OHS/ohs1/webgate/config/ to see if it got updated. After couple of minutes the following entry in this file
should get updated to a value of 0.
Now after getting all these things done, the problem has gone. All resources including IdM and Fusion Apps are working just fine. To confirm whether such approach is supported and correct, I have created a Service Request to Oracle, where I discussed that with both OAM and Fusion Apps engineers. First, the answers were quite confusing – as per the OAM team, that should not be an issue and usually it is not required to change the setting for OAM/Webgate deployments. However, when it comes to Fusion Apps and what aligns with answer from Fusion team is that this setting should be set to 0 and this is how it is being done in Oracle internal Fusion Apps environments. Although this is very interesting why Oracle has not added such valuable information to EDG or Release notes for Fusion Applications.
It was quite challenging for me to find and resolve the issue, so I have requested Oracle at least to add this to knowledge base to help another customers avoiding this. Luckily the answer was: “Hi Andrejs, I have created the following note for this issue, it will be reviewed and published Received Error 404- Not Found When Trying to Access Fusion Application Protected URL’s (Doc ID 1520998.1)“. So let’s hope it will appear in MOS soon.
This is a valuable piece of information. I have seen blogs and websites where they claim to have done installations and provide step by step but dont include updates like above.
Nice to see that some people like you are around and share good information.
Thank you for your comment. I was also struggling on finding this piece of information, therefore thought it would be valuable for people around doing bare metal Fusion Applications provisioning.
We followed the instructions you provided and we got past the 404 error.However upon restart we are getting the below error while accessing oamconsole/oim/weblogic console
System error. Please re-try your action. If you continue to get this error, please contact the Administrator.
The logs indicate the following error:
<Error occurred while handling the request.
java.lang.RuntimeException: No Event Factory provider configured for :check_authn_scheme
[APP: oam_server] [URI: /oam/server/obrareq.cgi] Error occurred while handling the request.[[
java.lang.RuntimeException: No Event Factory provider configured for :check_authn_scheme
We checked the integration steps for webgate-oam-oim. We dont seem to have missed any to best of our knowledge.
Could you advise on how we can get past this error or a pointer on where to look for ?
In addition, we found the following error:
privilege OAM11gApplication!GET resource //app/policy/OAM11gApplication/HTTP/af418c42e2d73486f8abe22ca4a98833a%COLON%%2Ffavicon%DOT%ico result DENY
Please advise if you can see anything obvious from the error.
It seems that OAM still denies access to unprotected URL’s.
Can you please double check the configuration as per the blog post and check that in Webgate configuration denyOnNotProtected is set as 0?
Alternatively, you should also increase Webgate logging level and check oblog.log. Do you see any error there? Let me know your findings.
I doublechecked and ound DenyonNotProtected unchecked for webgate. I also enabled the debug high on webgate and OAM and found only this error.
The debug log gives the following:
2013/03/30@06:01:49.00676 16478 16496 HTTP_REQ TRACE 0x00000204 /ade/aime_50584/ngamac/src/palantir/webgate2/src/redirect.cpp:664 “Function exited” _TraceName^WebGate::Redirect _TraceDuration^0.001214 eCond^FatalErr
We went ahead with the install and got the following error during post configure phase:
runProvisioning-postconfigure.out:2013-03-31 18:31:14.882 NOTIFICATION [logStatus] STATE=BUILD_ERROR!TIMESTAMP=2013-03-31 18:31:14 IST!TARGET=configure-ecsf!CATEGORY=BUILD_ERROR!DOMAIN=HCMDomain!HOSTNAME=hcm.fusion.com!PRODUCTFAMILY=hcm!PRODUCT=ATGPF!TASK=synchronized!TASKID=hcm.ATGPF.BUILD_ERROR.configure-ecsf.synchronized!MESSAGE=/u01/oracle/fusion/provisioning/provisioning-build/atgpf-build.xml:2678: The following error occurred while executing this line: /u01/oracle/fusion/provisioning/provisioning-build/common-misc-build.xml:109: ECSF configuration failed. Error message: Java returned: 1!
Have you seen this error ?
This is a known error. Are you installing 11.1.4 version? If you still have not resolved your issue, I can look up a solution for you.
We have not be able to resolve the error. We are installing 11.1.4 version. Please advise.
Can you check the following bug in MOS: Bug 13943173 : ERROR WHILE RUNNINGPOSTCONFIGURE ¿ECSF CONFIGURATION FAILED.
And the solution from Release notes: The workaround for the issue is:
Make the /tmp/_wl_proxy directory globally read/writable with the following UNIX command:
chmod 777 /tmp/_wl_proxy
If you have already begun the postconfiguration phase at the time of the error, retry the postconfiguration phase.
Also, you should look into runProvisioning-%-postconfigure.log log files – those should provide more clues on the error you are facing.
Hi Andrejs Karpovs
My name is Mariano, i am from Madrid, so sorry about my english, i will try to explain properly.
I found the related problem and follow your instructions in order to solve the problem and it worked perfectly. Anyway i have been investigating why this is happening, and found that denyOnNotProtected param, is set to “1” on the Request file sample (OAMRequest.xml), so if you take that file as a template and dont pay attention on that param, you will send it to OAM:
[oracle@OAMHOST input]$ pwd
[oracle@OAMHOST input]$ grep denyOnNotProtected OAMRequest.xml
So if you change that param before doing the registration, the problem dont appears.
I hope this information is correct and help others to go ahead
Thank you very much
Thanks for your comment. I am glad that you’ve found another workaround for this. Anyway, it seems no longer actual with 11.1.7 version, where everything is done automatically.
I am new to fusion environment and have been given a environment to maintain it. The issue is whenever i am logging out the Weblogic console page it is throwing 404 error.
Error 404–Not Found
From RFC 2068 Hypertext Transfer Protocol — HTTP/1.1:
10.4.5 404 Not Found
The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent.
If the server does not wish to make this information available to the client, the status code 403 (Forbidden) can be used instead. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address.
Earlier it was not showing the same and to surprise we have not done any change in the environmet because it has not started for using as of now..
Can anyone please help on the same
Great job! Saved the day!
To me, my OHS configuration needed to add some entries so it goes back to weblogic for unsecured stuff, and I used Doc ID 1327863.1 .
This was really helpful. Thank you.
Have this in webgate.conf of OHS instance.