Issues & resolutions


Issue # 1

<Sep 7, 2011 11:16:20 PM EDT> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: User weblogic is not permitted to boot the server; The server policy may have changed in such a way that the user is no longer able to boot the server.Reboot the server with the administrative user account or contact the system administrator to update the server policy definitions.
weblogic.security.SecurityInitializationException: User weblogic is not permitted to boot the server; The server policy may have changed in such a way that the user is no longer able to boot the server.Reboot the server with the administrative user account or contact the system administrator to update the server policy definitions.
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:1009)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1050)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:875)
at weblogic.security.SecurityService.start(SecurityService.java:141)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
Truncated. see log file for complete stacktrace
>
<Sep 7, 2011 11:16:20 PM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Sep 7, 2011 11:16:20 PM EDT> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Sep 7, 2011 11:16:20 PM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
---------------------------------------------

###########################
#    RESOLUTION      #
###########################

Delete state.chk, &  contents of policyA & policyB directories from work/runtime directory

Login into OES Admin console; go to Identity and select identity.dir (SCM) click on "edituser" reset password with domain weblogic password.

Distribute policies. and restart Admin JVM.

###########################
#    WORK AROUND      #
###########################

Make a backup of the config/config.xml file,
Switch the default security realm back to myrealm
run startWebLogic.sh and boot your domain

Adjust the WebLogic Security Providers

Log into console create the DefaultAuthenticator and the DefaultIdentityAsserter.
Set the JAAS Control Flag on both the DefaultAuthenticator and the DatabaseAuthenticator to SUFFICIENT and order the DefaultAuthenticator first

Switch the default security realm back to OES
Edit config.xml again, change default-realm back to what was there originally.


Boot your domain

Issue # 2


Delay in distributing Policies/Permissions while creating new ARME for more than 15 minutes.

###########################
#    RESOLUTION      #
###########################

Verify if /tmp/ALESXXXXXXX/XXX.log to see if another distribution is in progress.
if so run ./removeDistributionLock.sh in <BEA_HOME>/alesadmin/bin
Posted by Krishna Kasibatla at 12:47 PM No comments:
Email ThisBlogThis!Share to TwitterShare to Facebook
Labels: Issues
2011-08-18 16:43:50,831 [[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] WARN  com.bea.security.providers.authorization.asi.RbacAttributeRetriever - getAllSupportedAttributes(): Can not get IdentityQuery object. Make sure your metadirectory is properly configured.
<Aug 18, 2011 4:43:50 PM EDT> <Error> <Security> <BEA-090870> <The realm "qaliverealm" failed to be loaded: weblogic.security.service.SecurityServiceException: com.bea.common.engine.ServiceInitializationException: java.lang.IllegalArgumentException: Invalid signature length: -1..weblogic.security.service.SecurityServiceException: com.bea.common.engine.ServiceInitializationException: java.lang.IllegalArgumentException: Invalid signature length:
-1.
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealm(CommonSecurityServiceManagerDelegateImpl.java:465)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadRealm(CommonSecurityServiceManagerDelegateImpl.java:840)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealms(CommonSecurityServiceManagerDelegateImpl.java:869)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1030)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:881)
        Truncated. see log file for complete stacktrace

Caused By: com.bea.common.engine.ServiceInitializationException: java.lang.IllegalArgumentException: Invalid signature length: -1.
        at com.bea.common.engine.internal.ServiceEngineImpl.findOrStartService(ServiceEngineImpl.java:365)
        at com.bea.common.engine.internal.ServiceEngineImpl.findOrStartService(ServiceEngineImpl.java:315)
        at com.bea.common.engine.internal.ServiceEngineImpl.lookupService(ServiceEngineImpl.java:257)
        at com.bea.common.engine.internal.ServicesImpl.getService(ServicesImpl.java:72)
        at weblogic.security.service.CSSWLSDelegateImpl.getService(CSSWLSDelegateImpl.java:156)
        Truncated. see log file for complete stacktrace

Caused By: java.lang.IllegalArgumentException: Invalid signature length: -1.
        at com.bea.security.utils.SignatureUtils.readSignature(SignatureUtils.java:633)
        at com.bea.security.utils.SignatureUtils.verify(SignatureUtils.java:446)
        at com.bea.security.providers.authorization.asi.ARME.engine.UpdateManager.verifySignature(UpdateManager.java:1024)
        at com.bea.security.providers.authorization.asi.ARME.engine.UpdateManager.<init>(UpdateManager.java:170)
        at com.bea.security.providers.authorization.asi.ARME.engine.UpdateManager.getInstance(UpdateManager.java:191)
        Truncated. see log file for complete stacktrace
>
<Aug 18, 2011 4:43:50 PM EDT> <Notice> <Security> <BEA-090082> <Security initializing using security realm qaliverealm.>
<Aug 18, 2011 4:43:50 PM EDT> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

###########################
#    RESOLUTION      #
###########################

Reinstall only ARME's and start the domain servers.

###########################
# ISSUE    #
###########################
Delay in distributing Polcies/Permissions when creating new ARME for more than 15 minutes.

###########################
# RESOLUTION   #
###########################
Verify if /tmp/ALESXXXXXXX/XXX.log to see if another distribution is in progress.
if so run ./removeDistributionLock.sh in <BEA_HOME>/alesadmin/bin

###########################
# ISSUE    #
###########################

<Sep 7, 2011 11:16:20 PM EDT> <Critical> <Security> <BEA-090404> <User weblogic is not permitted to boot the server; The server policy may have changed in such a way that the user is no longer able to boot the server.Reboot the server with the administrative user account or contact the system administrator to update the server policy definitions.>
<Sep 7, 2011 11:16:20 PM EDT> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: User weblogic is not permitted to boot the server; The server policy may have changed in such a way that the user is no longer able to boot the server.Reboot the server with the administrative user account or contact the system administrator to update the server policy definitions.
weblogic.security.SecurityInitializationException: User weblogic is not permitted to boot the server; The server policy may have changed in such a way that the user is no longer able to boot the server.Reboot the server with the administrative user account or contact the system administrator to update the server policy definitions.
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:1009)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1050)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:875)
at weblogic.security.SecurityService.start(SecurityService.java:141)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
Truncated. see log file for complete stacktrace
>
<Sep 7, 2011 11:16:20 PM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Sep 7, 2011 11:16:20 PM EDT> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Sep 7, 2011 11:16:20 PM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
---------------------------------------------

###########################
# RESOLUTION   #
###########################
Delete state.chk, &  contents of policyA & policyB directories from work/runtime directory
Login into OES Admin console; go to Identity and select identity.dir (SCM) click on "edituser" reset password with domain weblogic password.
Distribute policies. and restart Admin JVM.

###########################
# WORK AROUND   #
###########################

Make a backup of the config/config.xml file,
Switch the default security realm back to myrealm
run startWebLogic.sh and boot your domain
Adjust the WebLogic Security Providers
Log into console create the DefaultAuthenticator and the DefaultIdentityAsserter.
Set the JAAS Control Flag on both the DefaultAuthenticator and the DatabaseAuthenticator to SUFFICIENT and order the DefaultAuthenticator first
Switch the default security realm back to OES
Edit config.xml again, change default-realm back to what was there originally.

Boot your domain

Switch the default security realm back to myrealm
Make a backup of the config/config.xml file, then edit the original  text editor. Search for the line that contains default-realm and change the value from whatever it currently says to myrealm but take careful note of the value you have there as you will need it later.
Boot your domain
run startWebLogic.sh/.bat and boot your domain
Adjust the WebLogic Security Providers
    Next, log into the weblogic console http://localhost:7001/console, and create the DefaultAuthenticator and the DefaultIdentityAsserter. Set the JAAS Control Flag on both the DefaultAuthenticator and the DatabaseAuthenticator to SUFFICIENT and order the DefaultAuthenticator first (I'll explain why in a second).
    Oh, so the reason for having to have a specific ordering on the authentication providers, and having the DefaultAuthenticator first and sufficient is that the DatabaseAuthenticator adds a special IdentityDirectoryPrincipal, that JDeveloper doesn't have the classes - so I simplified this by "tweaking" the realm. In practice, the DatabaseAuthenticator is not really used, but it is the authentication provider that is created by default in the configtool. This is really just a minor issue, but I wanted people to understand why the change.

If you're going to use JDeveloper to deploy your app to the domain then you should follow his instructions. If you've already deployed the application to the domain or don't plan to use JDeveloper to do that you can go ahead and plug in an LDAP Authenticator instead.
In any case remember to set the order properly and make the DefaultAuthenticator sufficient if you use it. Once you've made the changes shut the WebLogic Server down.
Switch the default security realm back to OES
Edit config.xml again, being sure to reload it from disk to pick up the changes we made a moment ago. Again find the default-realm and change the value back to what was there originally.
Boot your domain
Run startWebLogic again and the domain should startup normally. Wait until you see it reach the RUNNING state
####################
## DST UAT Issue ###
####################
2013-03-13 13:42:31,192 [[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR com.bea.security.ent
itlements.admin.beans.SessionState - no policy information is available for specified id
com.bea.ales.management.exception.ManagementException: no policy information is available for specified id
        at com.bea.ales.management.PolicyDistributor.handleException(PolicyDistributor.java:179)
        at com.bea.ales.management.PolicyDistributor.getDistributionStatus(PolicyDistributor.java:165)
        at com.bea.security.entitlements.admin.beans.SessionState.DistributionStatusDialogRefresh(SessionState.java:4320)
        at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
:

###########################
# RESOLUTION   #
###########################
shutdown admin, scm on server 1
./removedistributionlock.sh
start second server scm. admin
redistribute recent policies on 2nd policy server. once it is 100% done
log out of asi console; shutdown admin & scm.
./removedistributionLock.sh
start server 1 scm & admin; try to distribute policies

#################################################
## Initial set of POLICIES Distribution Issue ###
#################################################
Add below comments to start Admin to get Initial set of policies in case it doesnt fetches on its own. Once the policies are loaded can comment them back

#WLES_JAVA_OPTIONS="$WLES_JAVA_OPTIONS -Dcom.bea.security.providers.authorization.asi.AuthorizationProviderImpl.discoverymode=true"
#WLES_JAVA_OPTIONS="$WLES_JAVA_OPTIONS -Dcom.bea.security.providers.authorization.asi.RoleProviderImpl.discoverymode=true"

#################################################
Distribute POLICIES Issue                 
#################################################
SELECT * FROM DISTRIBUTION;
SELECT * FROM ENGINE_ADDRESSES;
select * from distribution where POLICY_NO = 0

select * from ENGINE_ADDRESSES where POLICY_ID = -10000
(it is for out f sync)
select * from ENGINE_ADDRESSES where POLICY_ID = -1
(it is for full distribution)

Enable Logging.
log4j.logger.com.bea.security.psl.jdo.BaseJdoImpl=DEBUG
log4j.logger.com.bea.security.psl.jdo.SubjectManagerJdoImpl=DEBUG
log4j.logger.com.bea.security.psl.jdo.UserManagerJdoImpl=DEBUG
log4j.logger.com.bea.security.pdws=DEBUG
log4j.logger.com.bea.security.pdws=DEBUG
log4j.logger.com.bea.security.pdws.psl.computing=DEBUG
log4j.logger.com.bea.security.pdws.psl=DEBUG

If the LOCK persists though distribution is not taking place and if its not released even with removedistributionLock.sh script.

Check for unsynchronized list of SSM's in asi console.
If all/most of them are unsynchronized then shutdown all JVM's (SSM's) .
Delete Unsynchronized list of SSM's
Restart SCM, Admin and followed by each JVM (SSM)

Make sure this time SSM comes into Synchronized mode.

No comments:

Post a Comment