IIS 7.5 Server behind an F5 Reverse Proxy

[Meiselman, Ellen]

Hi,

I installed shibboleth on an IIS server - call it contentserver.com<http://contentserver.com>. Everything was working.

Then we put that server behind an F5 reverse proxy - let's call its hostname "proxyserver.com<http://proxyserver.com>"

So now, pointing the browser to https://proxyserver.com will display content that is actually on https://contentserver.com

How do I set up the SSL certs so shibboleth will work for requests that come in to https://proxyserver.com? I have only very basic knowledge of how to install and bind certs.

Right now I can't get Shibboleth working again. The plugin is up and running but I can't get the appropriate metadata from the IDP because I can't give them the correct certificate - at least I don't *think* I can. I don't know how to install a certificate that is for another hostname.


______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org<mailto:emeiselm-63aXycvo3TyHXe+***@public.gmane.org>
Phone [phone]

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

On 8/19/14, 11:38 AM, "Meiselman, Ellen" <emeiselm-mMBBrLdnlFaUArH+***@public.gmane.org<mailto:emeiselm-mMBBrLdnlFaUArH+***@public.gmane.org>> wrote:

How do I set up the SSL certs so shibboleth will work for requests that come in to
https://proxyserver.com? I have only very basic knowledge of how to install and bind certs.

The SSL certs are unrelated to Shibboleth, that's just basic web site set up. I have no idea how you're planning t deploy it, there could be SSL on one or both legs, and the certs are going to be different on the two layers, but none of them are used by the SP.

Right now I can't get Shibboleth working again. The plugin is up and running but I can't get the appropriate metadata from the IDP because I can't give them the correct certificate - at least I don't *think* I can. I don't know how to install a certificate that is for another hostname.

The certificate in the SP has nothing to do with SSL server use, and doesn't change here whether there's a proxy or not.

And, you don't get metadata from an IdP by authenticating with a certificate. Metadata is signed and generally made available on a web site, it's not protected. If it is, that's a matter for you and your administrator I suppose.

-- Scott

[Christopher Bongaarts]

Post by Meiselman, Ellen
I installed shibboleth on an IIS server - call it contentserver.com
<http://contentserver.com>. Everything was working.
Then we put that server behind an F5 reverse proxy - let's call its
hostname "proxyserver.com <http://proxyserver.com>"
So now, pointing the browser to *https://proxyserver.com* will display
content that is actually on https://contentserver.com
How do I set up the SSL certs so shibboleth will work for requests
that come in to https://proxyserver.com? I have only very basic
knowledge of how to install and bind certs.
Right now I can't get Shibboleth working again. The plugin is up and
running but I can't get the appropriate metadata from the IDP because
I can't give them the correct certificate - at least I don't *think* I
can. I don't know how to install a certificate that is for another
hostname.

Keep in mind that there are usually two sets of certificates in play
here - an SSL certificate that is used to secure your HTTPS traffic, and
a SAML encryption certificate that is used by Shibboleth.

Your SSL certificate is configured in the web server (IIS GUI in your
case). This is the one that you typically buy from a public certificate
authority.

The Shib certificate (and corresponding private key) is normally
generated at installation time and lives in the Shib configuration
directory alongside shibboleth2.xml. Default name "shib-sp.pem" I
think. The certificate details are not used at all by the Shib software
- the server name, expiration, CA signatures etc. are all irrelevant.
It is just used as a convenient way to carry a public key. The Shib
certificate is the one that is included in your SP's metadata file (that
you give to your IdP).

Also in the SP metadata file are several SAML endpoints (the XML
elements that contain a Location attribute). They tell the IdP how to
direct SAML messages to your SP. If you've changed the publicly-facing
name of the server, you'll also need to update these endpoints so they
contain the correct URLs (as viewed by the browser). Since you're on
IIS, you'll also need to verify that your Site directives in your
shibboleth2.xml file are correct.

If you use the metadata generator endpoint (/Shibboleth.sso/Metadata),
it will include the currently configured certificate and whatever
virtual hostname was used to access it in the endpoints.

In all likelihood, you've left your Shib certificates alone, so in
theory the only thing you should need to do (on the Shib side) is change
the Locations of the endpoints in your metadata, and ask the IdP to use
the updated metadata.

[Meiselman, Ellen]

Thank you Chris - this was extremely helpful. You stopped me from going down a dead end. I checked the Native error logs and found it was giving this error:

ERROR Shibboleth.ISAPI [3820] isapi_shib: Unable to locate metadata for identity provider

Which to me means a bad path tot he IdP metadata file, most likely. I then revisited the shibboleth.xml settings file and corrected the relative paths to /Metadata, /Session, etc. since they now have the reverse proxy-necessitated directory /content/in the path. I added handlerURL=/content/Shibboleth.sso" to the Sessions element.

<Sessions lifetime="28800" timeout="3600" relayState="ss:mem" handlerURL="/content/Shibboleth.sso"
checkAddress="false" handlerSSL="true" cookieProps="https">

Now if I browse to
https://proxyserver.com/content/Shibboleth.sso/Metadata
I get this message from the IdP:

Error:Unable to complete request at this time. (Request was from an untrusted provider-31DE48E3C3946D8E)

Which is a whole lot farther along than before.

Now here is where I am confused still. The metadata.xml file in /etc/ was given me by my IdP, and was at least apparently working before I changed hostnames. The only locations it contains all refer to the IdP. So that is the Identity Provider metadata, correct?

I tried the endpoint .../Shibboleth.sso/Metadata on another content server I have that is functional. That endpoint gives me what looks like it must be SP metadata. Is that xml file entirely generated by Shibboleth according to some template? Or have I just not found the file yet? What controls what goes into that file?



Thank you very much for your help!

Ellen
______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org<mailto:emeiselm-63aXycvo3TyHXe+***@public.gmane.org>
Phone [phone]

On Aug 19, 2014, at 12:04 PM, Christopher Bongaarts <cab-***@public.gmane.org<mailto:***@umn.edu>> wrote:

On 8/19/2014 10:38 AM, Meiselman, Ellen wrote:
I installed shibboleth on an IIS server - call it contentserver.com<http://contentserver.com>. Everything was working.

Then we put that server behind an F5 reverse proxy - let's call its hostname "proxyserver.com"

So now, pointing the browser to https://proxyserver.com will display content that is actually on https://contentserver.com

How do I set up the SSL certs so shibboleth will work for requests that come in to https://proxyserver.com? I have only very basic knowledge of how to install and bind certs.

Right now I can't get Shibboleth working again. The plugin is up and running but I can't get the appropriate metadata from the IDP because I can't give them the correct certificate - at least I don't *think* I can. I don't know how to install a certificate that is for another hostname.


Keep in mind that there are usually two sets of certificates in play here - an SSL certificate that is used to secure your HTTPS traffic, and a SAML encryption certificate that is used by Shibboleth.

Your SSL certificate is configured in the web server (IIS GUI in your case). This is the one that you typically buy from a public certificate authority.

The Shib certificate (and corresponding private key) is normally generated at installation time and lives in the Shib configuration directory alongside shibboleth2.xml. Default name "shib-sp.pem" I think. The certificate details are not used at all by the Shib software - the server name, expiration, CA signatures etc. are all irrelevant. It is just used as a convenient way to carry a public key. The Shib certificate is the one that is included in your SP's metadata file (that you give to your IdP).

Also in the SP metadata file are several SAML endpoints (the XML elements that contain a Location attribute). They tell the IdP how to direct SAML messages to your SP. If you've changed the publicly-facing name of the server, you'll also need to update these endpoints so they contain the correct URLs (as viewed by the browser). Since you're on IIS, you'll also need to verify that your Site directives in your shibboleth2.xml file are correct.

If you use the metadata generator endpoint (/Shibboleth.sso/Metadata), it will include the currently configured certificate and whatever virtual hostname was used to access it in the endpoints.

In all likelihood, you've left your Shib certificates alone, so in theory the only thing you should need to do (on the Shib side) is change the Locations of the endpoints in your metadata, and ask the IdP to use the updated metadata.
--
%% Christopher A. Bongaarts %%
cab-***@public.gmane.org<mailto:cab-***@public.gmane.org>
%%
%% OIT - Identity Management %%
http://umn.edu/~cab
%%
%% University of Minnesota %% [phone] %%

--
To unsubscribe from this list send an email to users-***@shibboleth.net<mailto:users-unsubscribe-***@public.gmane.org>

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

Post by Meiselman, Ellen
Which to me means a bad path tot he IdP metadata file, most likely. I
then revisited the shibboleth.xml settings file and corrected the
relative paths to /Metadata, /Session, etc. since they now have the
reverse proxy-necessitated directory /content/in the
path. I added handlerURL=/content/Shibboleth.sso" to the Sessions
element.

Well, you can do that, but I really don't advise that unless you
absolutely have to. Your reverse proxy should map the entire vhost to your
entire vhost. Playing games with that will make your life infinitely
harder.

Post by Meiselman, Ellen
Now if I browse to
https://proxyserver.com/content/Shibboleth.sso/Metadata

That URL should never involve the IdP so you have a very fundamental
problem going on. Your system does not know about the virtualization
you're attempting because it doesn't think the request is for a handler
inside the SP.

Post by Meiselman, Ellen
Now here is where I am confused still. The metadata.xml file in /etc/ was
given me by my IdP, and was at least apparently working before I changed
hostnames. The only locations it contains all refer to the IdP. So that
is the Identity Provider metadata, correct?

Yes.

Post by Meiselman, Ellen
I tried the endpoint .../Shibboleth.sso/Metadata on another content
server I have that is functional. That endpoint gives me what looks like
it must be SP metadata. Is that xml file
entirely generated by Shibboleth according to some template? Or have I
just not found the file yet? What controls what goes into that file?

It's generated on the fly to provide examples for people to work on. Your
system is broken so when you access it, it's dispatching to the IdP
apparently, at least based on what you said. That means the SP doesn't
believe that the request is for a handler, but for content.

That means it's not seeing requests come in with the values you think it
is. Turn up logging in native.log and see what the RequestMap is seeing on
DEBUG.

-- Scott

[Meiselman, Ellen]

Thanks Scott, my answers are below.

Post by Cantor, Scott

Post by Meiselman, Ellen
Which to me means a bad path tot he IdP metadata file, most likely. I
then revisited the shibboleth.xml settings file and corrected the
relative paths to /Metadata, /Session, etc. since they now have the
reverse proxy-necessitated directory /content/in the
path. I added handlerURL=/content/Shibboleth.sso" to the Sessions
element.

Well, you can do that, but I really don't advise that unless you
absolutely have to. Your reverse proxy should map the entire vhost to your
entire vhost. Playing games with that will make your life infinitely
harder.

They can't, unfortunately. The root of the domain proxyserver.com is being used for the LMS application. The /content/ subdirectory is being used for our SCORM content which has to use the same domain, protocol, port, etc. so that javascript can communicate across frames.

Post by Cantor, Scott

Post by Meiselman, Ellen
Now if I browse to
https://proxyserver.com/content/Shibboleth.sso/Metadata

That URL should never involve the IdP so you have a very fundamental
problem going on. Your system does not know about the virtualization
you're attempting because it doesn't think the request is for a handler
inside the SP.

OK... I will keep that in mind. Not sure yet what to do to fix it.

Post by Cantor, Scott

Post by Meiselman, Ellen
Now here is where I am confused still. The metadata.xml file in /etc/ was
given me by my IdP, and was at least apparently working before I changed
hostnames. The only locations it contains all refer to the IdP. So that
is the Identity Provider metadata, correct?

Yes.

Post by Meiselman, Ellen
I tried the endpoint .../Shibboleth.sso/Metadata on another content
server I have that is functional. That endpoint gives me what looks like
it must be SP metadata. Is that xml file
entirely generated by Shibboleth according to some template? Or have I
just not found the file yet? What controls what goes into that file?

It's generated on the fly to provide examples for people to work on. Your
system is broken so when you access it, it's dispatching to the IdP
apparently, at least based on what you said. That means the SP doesn't
believe that the request is for a handler, but for content.

So that means I still probably have a path problem with my setting for the metadata handler.

Post by Cantor, Scott
That means it's not seeing requests come in with the values you think it
is. Turn up logging in native.log and see what the RequestMap is seeing on
DEBUG.

OK, thank you I will!!

Post by Cantor, Scott
-- Scott
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

Post by Meiselman, Ellen
They can't, unfortunately. The root of the domain proxyserver.com is
being used for the LMS application. The /content/ subdirectory is being
used for our SCORM content which has to use the same domain, protocol,
port, etc. so that javascript can communicate across frames.

The first mistake was really using frames then, they should be banned at
this point.

Post by Meiselman, Ellen
So that means I still probably have a path problem with my setting for the metadata handler.

No, it means your system doesn't know what it's named and how it's being
accessed. When you virtualize a system, the web server needs extensive
features to support that. IIS has none of them, so at a basic level it
does not support load balancers. If you can't tell a web server that its
logical settings are different from its physical settings, then you can't
virtualize it unless all the applications running on it are themselves
broken and ignore the web server's own reporting of the values to
applications.

The SP compensates for that imperfectly through the Site mappings in its
own configuration, and you haven't set those mappings up correctly to
report the virtualized scheme, port, or hostname, or all three.

The metadata handler is irrelevant anyway, it's for providing example
metadata, not for production use. None of your handlers are going to work,
and the assertion consumer service not working means the SP doesn't work
period.

-- Scott

[Meiselman, Ellen]

Post by Cantor, Scott

Post by Meiselman, Ellen
They can't, unfortunately. The root of the domain proxyserver.com is
being used for the LMS application. The /content/ subdirectory is being
used for our SCORM content which has to use the same domain, protocol,
port, etc. so that javascript can communicate across frames.

The first mistake was really using frames then, they should be banned at
this point.

Agreed, but this is an old Elearning standard, soon to be replaced by the xAPI. However we have to make this work for now.

Post by Cantor, Scott

Post by Meiselman, Ellen
So that means I still probably have a path problem with my setting for
the metadata handler.

No, it means your system doesn't know what it's named and how it's being
accessed. When you virtualize a system, the web server needs extensive
features to support that. IIS has none of them, so at a basic level it
does not support load balancers. If you can't tell a web server that its
logical settings are different from its physical settings, then you can't
virtualize it unless all the applications running on it are themselves
broken and ignore the web server's own reporting of the values to
applications.
The SP compensates for that imperfectly through the Site mappings in its
own configuration, and you haven't set those mappings up correctly to
report the virtualized scheme, port, or hostname, or all three.
The metadata handler is irrelevant anyway, it's for providing example
metadata, not for production use. None of your handlers are going to work,
and the assertion consumer service not working means the SP doesn't work
period.

OK, thanks - I realize this is a non-optimal setup. I'll try to get the mappings right.

Post by Cantor, Scott
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Meiselman, Ellen]

Hi,

Scott, thank you for your previous help. I got the handlers (Metadata, Session, etc) working thanks to your suggestions. It now redirects to the login screen as it should.

To recap: I have an SP behind a reverse proxy that has mapped a path (https://proxyserver.com/content/) to the root of our IIS webserver (https://contentserver.com).

Browsing to https://proxyserver.com/content/somepage.htm now redirects to the SAML login screen as it should.

The problem: Upon logging in, it goes into a loop. Looping behavior that I've seen before was always caused by mismatched https/http cookie issues, but I haven't been able to figure out what the cause is here yet.

I need help with the best practice for debugging the looping behavior. I'm not sure what to look for in particular.

Right now my settings in shibboleth2.xml are:

<Site id="1" name="proxyserver.com/content" />

------

<RequestMap>
<Host name="proxyserver.com/content" authType="shibboleth" requireSession="true"/>
</RequestMap>
------

<ApplicationDefaults entityID="https://proxyserver.com/content" homeURL="https://proxyserver.com/content/iisstart.htm"
REMOTE_USER="eppn persistent-id targeted-id">

<Sessions lifetime="28800" timeout="3600" relayState="ss:mem" handlerURL="/content/Shibboleth.sso"
checkAddress="false" handlerSSL="true" cookieProps="https">

------

<!-- Extension service that generates "approximate" metadata based on SP configuration. -->
<Handler type="MetadataGenerator" Location="/Metadata" signing="false"/>

<!-- Status reporting service. -->
<Handler type="Status" Location="/Status" acl="127.0.0.1 ::1"/>

<!-- Session diagnostic service. -->
<Handler type="Session" Location="/Session" showAttributeValues="true"/>

<!-- JSON feed of discovery information. -->
<Handler type="DiscoveryFeed" Location="/DiscoFeed"/>


Thank you very much for your help.

Ellen

______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org
Phone [phone]

Post by Cantor, Scott

Post by Meiselman, Ellen
They can't, unfortunately. The root of the domain proxyserver.com is
being used for the LMS application. The /content/ subdirectory is being
used for our SCORM content which has to use the same domain, protocol,
port, etc. so that javascript can communicate across frames.

The first mistake was really using frames then, they should be banned at
this point.

Post by Meiselman, Ellen
So that means I still probably have a path problem with my setting for
the metadata handler.

No, it means your system doesn't know what it's named and how it's being
accessed. When you virtualize a system, the web server needs extensive
features to support that. IIS has none of them, so at a basic level it
does not support load balancers. If you can't tell a web server that its
logical settings are different from its physical settings, then you can't
virtualize it unless all the applications running on it are themselves
broken and ignore the web server's own reporting of the values to
applications.
The SP compensates for that imperfectly through the Site mappings in its
own configuration, and you haven't set those mappings up correctly to
report the virtualized scheme, port, or hostname, or all three.
The metadata handler is irrelevant anyway, it's for providing example
metadata, not for production use. None of your handlers are going to work,
and the assertion consumer service not working means the SP doesn't work
period.
-- Scott
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Meiselman, Ellen]

I found the page on "looping" in the wiki just now - I'll give that a read and see if it gets me a little farther.

Ellen
______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org
Phone [phone]

Post by Meiselman, Ellen
Hi,
Scott, thank you for your previous help. I got the handlers (Metadata, Session, etc) working thanks to your suggestions. It now redirects to the login screen as it should.
To recap: I have an SP behind a reverse proxy that has mapped a path (https://proxyserver.com/content/) to the root of our IIS webserver (https://contentserver.com).
Browsing to https://proxyserver.com/content/somepage.htm now redirects to the SAML login screen as it should.
The problem: Upon logging in, it goes into a loop. Looping behavior that I've seen before was always caused by mismatched https/http cookie issues, but I haven't been able to figure out what the cause is here yet.
I need help with the best practice for debugging the looping behavior. I'm not sure what to look for in particular.
<Site id="1" name="proxyserver.com/content" />
------
<RequestMap>
<Host name="proxyserver.com/content" authType="shibboleth" requireSession="true"/>
</RequestMap>
------
<ApplicationDefaults entityID="https://proxyserver.com/content" homeURL="https://proxyserver.com/content/iisstart.htm"
REMOTE_USER="eppn persistent-id targeted-id">
<Sessions lifetime="28800" timeout="3600" relayState="ss:mem" handlerURL="/content/Shibboleth.sso"
checkAddress="false" handlerSSL="true" cookieProps="https">
------
<!-- Extension service that generates "approximate" metadata based on SP configuration. -->
<Handler type="MetadataGenerator" Location="/Metadata" signing="false"/>
<!-- Status reporting service. -->
<Handler type="Status" Location="/Status" acl="127.0.0.1 ::1"/>
<!-- Session diagnostic service. -->
<Handler type="Session" Location="/Session" showAttributeValues="true"/>
<!-- JSON feed of discovery information. -->
<Handler type="DiscoveryFeed" Location="/DiscoFeed"/>
Thank you very much for your help.
Ellen
______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
Phone [phone]

Post by Cantor, Scott

Post by Meiselman, Ellen
They can't, unfortunately. The root of the domain proxyserver.com is
being used for the LMS application. The /content/ subdirectory is being
used for our SCORM content which has to use the same domain, protocol,
port, etc. so that javascript can communicate across frames.

The first mistake was really using frames then, they should be banned at
this point.

Post by Meiselman, Ellen
So that means I still probably have a path problem with my setting for
the metadata handler.

No, it means your system doesn't know what it's named and how it's being
accessed. When you virtualize a system, the web server needs extensive
features to support that. IIS has none of them, so at a basic level it
does not support load balancers. If you can't tell a web server that its
logical settings are different from its physical settings, then you can't
virtualize it unless all the applications running on it are themselves
broken and ignore the web server's own reporting of the values to
applications.
The SP compensates for that imperfectly through the Site mappings in its
own configuration, and you haven't set those mappings up correctly to
report the virtualized scheme, port, or hostname, or all three.
The metadata handler is irrelevant anyway, it's for providing example
metadata, not for production use. None of your handlers are going to work,
and the assertion consumer service not working means the SP doesn't work
period.
-- Scott
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

Post by Meiselman, Ellen
I need help with the best practice for debugging the looping behavior.
I'm not sure what to look for in particular.

Step 0: Check all logs and rule out the cookie being sent at the end but
the session being invalidated for some reason. Assuming no sign of that,
the cookie's not sent.

Step 1: Read, and fully grasp, the flows and cookies used [1].

Step 2: Trace all the traffic and find the spot where the cookie is
dropped.

-- Scott

[1] https://wiki.shibboleth.net/confluence/display/SHIB2/FlowsAndConfig

[Meiselman, Ellen]

OK, thank you.
______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org
Phone [phone]

Post by Cantor, Scott

Post by Meiselman, Ellen
I need help with the best practice for debugging the looping behavior.
I'm not sure what to look for in particular.

Step 0: Check all logs and rule out the cookie being sent at the end but
the session being invalidated for some reason. Assuming no sign of that,
the cookie's not sent.
Step 1: Read, and fully grasp, the flows and cookies used [1].
Step 2: Trace all the traffic and find the spot where the cookie is
dropped.
-- Scott
[1] https://wiki.shibboleth.net/confluence/display/SHIB2/FlowsAndConfig
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Meiselman, Ellen]

This weekend I located some issues with the SP metadata (http instead of https on some of the handler entries) but when that was corrected it still did not fix the looping. I've checked screenshots of the IdP's settings page for my SP, and I can't detect any issues with his settings.

So today we went through and examined every bit of the network requests and cookies from a login attempt.

We didn't find any smoking guns until we changed the relayState setting to "cookie" and inspected the value of the cookie. It turns out there is alway a double slash after the "content" part of the path:

https://proxyserver.com/content//whatever

for example:

https://proxyserver.com/content//Shibboleth.sso

https://proxyserver.com/content//mycontent.htm

There are absolutely no trailing slashes after the word "content" in shibboleth2.xml.


I am thinking this has something to do with the reverse proxy mapping settings or with my IIS settings. So I have two questions:

A. Do you think the extra slash in path in the cookie value might cause the looping problem?

B. Where would you start to look for the cause?


Thanks for your help,

Ellen



On Aug 22, 2014, at 7:08 PM, Cantor, Scott wrote:

On 8/22/14, 4:30 PM, "Meiselman, Ellen" <emeiselm-mMBBrLdnlFaUArH+***@public.gmane.org<mailto:emeiselm-mMBBrLdnlFaUArH+***@public.gmane.org>> wrote:

I need help with the best practice for debugging the looping behavior.
I'm not sure what to look for in particular.

Step 0: Check all logs and rule out the cookie being sent at the end but
the session being invalidated for some reason. Assuming no sign of that,
the cookie's not sent.

Step 1: Read, and fully grasp, the flows and cookies used [1].

Step 2: Trace all the traffic and find the spot where the cookie is
dropped.

-- Scott

[1] https://wiki.shibboleth.net/confluence/display/SHIB2/FlowsAndConfig

--
To unsubscribe from this list send an email to users-***@shibboleth.net<mailto:users-unsubscribe-***@public.gmane.org>

______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org<mailto:emeiselm-63aXycvo3TyHXe+***@public.gmane.org>
Phone [phone]

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

Post by Meiselman, Ellen
We didn't find any smoking guns until we changed the relayState setting
to "cookie" and inspected the value of the cookie. It turns out there is
There are absolutely no trailing slashes after the word "content" in shibboleth2.xml.

Well, it's your proxy doing it.

Post by Meiselman, Ellen
A. Do you think the extra slash in path in the cookie value might cause the looping problem?

I doubt it, but I guess it's possible. The issue isn't what the URLs are
per se, but where/when the session cookie is dropped. That should be
evident from a trace. Once the droppage is confirmed and on what step,
it's a given that there's a mismatch between where it's set and where's
it's not sent back. Then you can figure out which one is off.

Is there some value this reverse proxy is providing?

-- Scott

[Meiselman, Ellen]

Hi Scott,

I did some tracing with LiveHTTPHeaders and compared between working and non-working servers today. I think I've found the spot where things appear to go off the rails but am not certain what to do about it. We're going to set up a little in-house reverse proxy and experiment, since it is quite difficult to get the cloud vendor to change settings.

You said:

...The issue isn't what the URLs are
per se, but where/when the session cookie is dropped. That should be
evident from a trace. Once the droppage is confirmed and on what step,
it's a given that there's a mismatch between where it's set and where's
it's not sent back. Then you can figure out which one is off.

Here's what I have so far. I've boiled down the trace to what I think is most significant:
______________________________
Line 1. https://reverseproxy.com/content
GET /content HTTP/1.1
Host: mlearningqa.med.umich.edu<http://mlearningqa.med.umich.edu>

HTTP/1.1 302 Please Wait
Location: https://identityprovider.com:8443/nidp/saml2/sso?SAMLRequest=.......*&RelayState=cookie%3A1409080194_2900
Set-Cookie: _shibstate_1409080194_2900=https%3A%2F%2Freverseproxy.com%2Fcontent%2F; path=/; secure; HttpOnly
Set-Cookie: SABASESSIONID=1336989325.20480.0000; path=/ (this is a non-shibboleth-related cookie set by the Reverse Proxy)
______________________________
Line 23. https://identityprovider.com:8443/nidp/saml2/sso?SAMLRequest=....&RelayState=cookie%3A1409080194_2900
GET /nidp/saml2/sso?SAMLRequest=.....&RelayState=cookie%3A1409080194_2900 HTTP/1.1
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=BC18E99BDCF778DD44FD14B9044953DF; Path=/nidp/; Secure; HttpOnly
Set-Cookie: UrnNovellNidpClusterMemberId=~03~02fdf~1B~16~11uqr; Path=/nidp
Set-Cookie: urn:novell:nidp:cluster:member:id=~03~02fdf~1B~16~11uqr; Path=/nidp
______________________________
Line 44. https://identityprovider.com<http://identityprovider.com>:8443/nidp/saml2/sso?id=716&sid=0&option=credential&sid=0
POST /nidp/saml2/sso?id=716&sid=0&option=credential&sid=0 HTTP/1.1
Cookie: JSESSIONID=BC18E99BDCF778DD44FD14B9044953DF; UrnNovellNidpClusterMemberId=~03~02fdf~1B~16~11uqr; urn:novell:nidp:cluster:member:id=~03~02fdf~1B~16~11uqr
______________________________
Line 68. https://identityprovider.com<http://identityprovider.com>:8443/nidp/saml2/sso?sid=0
POST /nidp/saml2/sso?sid=0 HTTP/1.1
Cookie: JSESSIONID=BC18E99BDCF778DD44FD14B9044953DF; UrnNovellNidpClusterMemberId=~03~02fdf~1B~16~11uqr; urn:novell:nidp:cluster:member:id=~03~02fdf~1B~16~11uqr
______________________________
Line 91. https://identityprovider.com:8443/nidp/saml2/sso?sid=0
GET /nidp/saml2/sso?sid=0 HTTP/1.1
Cookie: JSESSIONID=BC18E99BDCF778DD44FD14B9044953DF; UrnNovellNidpClusterMemberId=~03~02fdf~1B~16~11uqr; urn:novell:nidp:cluster:member:id=~03~02fdf~1B~16~11uqr
______________________________
Line 112. https://reverseproxy.com/content/Shibboleth.sso/SAML2/POST
POST /content/Shibboleth.sso/SAML2/POST HTTP/1.1
Location: https://identityprovider.com:8443/nidp/saml2/sso?SAMLRequest=...&RelayState=cookie%3A1409080201_2348
Set-Cookie: _shibstate_1409080201_2348=https%3A%2F%2Freverseproxy.com%2Fcontent%2F%2FShibboleth.sso%2FSAML2%2FPOST; path=/; secure; HttpOnly
______________________________

Note that a new _shibstate cookie is set at this spot with a new number appended to its name. Its value is now /content//Shibboleth.sso/SAML/POST

The server that is NOT behind the RP does not do this. Instead it sets a cookie called _shibsession_ at this point and then goes off to the resource that was requested, but this server does not.



You asked:

Is there some value this reverse proxy is providing?

It is required to enable Javascript communication between frames which would normally be in different domains. This is a requirement of the SCORM standard. With our current in-house hosted LMS, we used virtual directories to get the same result - content in the same domain as the LMS.




One thing we noticed today in the wiki instructions for Reverse Proxy:

* Note that the path (/secure) to the requested resource is set by the Shibboleth SP and hence is specific to the protected resource on the web server. This mandates that the proxy either proxies the resource with the exact same path (/secure to /secure), or that the proxy is able to rewrite HTTP resonse headers (e.g. the ones containing the relayState) before returning results to the client.

https://wiki.shibboleth.net/confluence/display/SHIB2/SPReverseProxy

We're mapping /content to / (webroot) So that could be one problem at least.



_______________________________________________________________________________


On Aug 25, 2014, at 10:19 PM, Cantor, Scott wrote:

On 8/25/14, 7:18 PM, "Meiselman, Ellen" <emeiselm-mMBBrLdnlFaUArH+***@public.gmane.org<mailto:emeiselm-mMBBrLdnlFaUArH+***@public.gmane.org>> wrote:

We didn't find any smoking guns until we changed the relayState setting
to "cookie" and inspected the value of the cookie. It turns out there is
alway a double slash after the "content" part of the path:

There are absolutely no trailing slashes after the word "content" in
shibboleth2.xml.

Well, it's your proxy doing it.

A. Do you think the extra slash in path in the cookie value might cause
the looping problem?

I doubt it, but I guess it's possible. The issue isn't what the URLs are
per se, but where/when the session cookie is dropped. That should be
evident from a trace. Once the droppage is confirmed and on what step,
it's a given that there's a mismatch between where it's set and where's
it's not sent back. Then you can figure out which one is off.


Is there some value this reverse proxy is providing?

It is required to enable Javascript communication between frames which would normally be in different domains. This is a requirement of the SCORM standard. With our current in-house hosted LMS, we used virtual directories to get the same result - content in the same domain as the LMS.

In the new cloud system we are attempting to use a reverse proxy to achieve this. While we were looking at the wiki article on Reverse Proxy again today, a member of my team noticed this note:

https://wiki.shibboleth.net/confluence/display/SHIB2/SPReverseProxy
• Note that the path (/secure) to the requested resource is set by the Shibboleth SP and hence is specific to the protected resource on the web server. This mandates that the proxy either proxies the resource with the exact same path (/secure to /secure), or that the proxy is able to rewrite HTTP resonse headers (e.g. the ones containing the relayState) before returning results to the client.

We are not proxying to the identical path.

https://proxyserver.com/content

displays content at

https://contentserver.com




-- Scott

--
To unsubscribe from this list send an email to users-***@shibboleth.net

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

Post by Meiselman, Ellen
______________________________
Line 112. https://reverseproxy.com/content/Shibboleth.sso/SAML2/POST

For that to work, you'd have to have a handlerURL on the SP of
/content/Shibboleth.sso. In addition, the request to that server had
better be for /content/Shibboleth.sso/SAML2/POST. If it's not, the SP
won't handle it, and even if it did, it wouldn't match the URL that's
embedded inside the message from the IdP and will be rejected. If it's
/content on the front-end, it has to be /content on the back-end.

Post by Meiselman, Ellen
Note that a new _shibstate cookie is set at this spot with a new number
appended to its name. Its value is now /content//Shibboleth.sso/SAML/POST

That's because your SP doesn't believe that that is the location of its
handlers, *and* it is being told to protect all URLs with /content in
them. Combine the two and it sees that as a protected resource and issues
a request for a login. You don't need to see a loop to see that, just
access /content/Shibboleth.sso/Foo and when that redirects to the IdP, you
know you have an unworkable system.

The only way the SP will exempt URLs that match protection rules is if
they're for its handlers, and it determines that using the handlerURL
setting.

Post by Meiselman, Ellen
* Note that the path (/secure) to the requested resource is set by the
Shibboleth SP and hence is specific to the protected resource on the web
server. This mandates that the proxy either proxies the resource with the
exact same path (/secure to /secure), or that the proxy is able to
rewrite HTTP resonse headers (e.g. the ones containing
the relayState) before returning results to the client.
https://wiki.shibboleth.net/confluence/display/SHIB2/SPReverseProxy
We're mapping /content to / (webroot) So that could be one problem at least.

It is. You can't do that. The suggestion above won't help either. That
might work if the content URLs were the ones with the mismatched paths,
but you can't do that with the handlers. The message from the IdP will
contain the URL to the proxy, and if that includes a path, the same path
has to be in the request to the back-end server as seen by the SP or it's
going to reject the message.

You can rewrite cookies and RelayState, but you can't rewrite a signed
message.

So full stop, you can't do it.

-- Scott

[Meiselman, Ellen]

Hi Scott,

I thought I'd report back because we had unexpected success even with our mucked-up reverse proxy mapping. Sorry this is long but I am trying to explain exactly what we did. I have two questions at the end.



After your last email I put in a ticket to the cloud vendor to change our reverse proxy mapping to what is recommended. However any sort of change typically takes them 2 weeks, so we had time to experiment with the reverse proxy mapping the way it was.

To recap, the issue was that because we have a mapping from a path "/content/" on the reverse proxy server to the root of our SP content server, the paths in the signed statement would include the path /content/ which did not exist on the back end SP server.

https://proxyserver.com/content/
maps to
https://contentserver.com



So, we tried fixing the problem from the back end which is the only side we can experiment with quickly.

We added a virtual directory called "/content/" to the SP content server which pointed to a real directory at the root of the server then added another directory inside it "ct" that points to the resources directory.

The virtual directory /content/ on the content server points to \webroot\directory\

The virtual directory /content/ct/ points to E:\path\to\learningmodules\

At that point the looping stopped immediately for the first time. However it was not entirely working.

When I would try to access a resource:

https://proxyserver.com/content/ct/resource.htm

it would redirect to the web root
https://proxyserver.com/content/

So I assumed it wasn't working. Tried clearing cookies, restarting browsers, restarting IIS, restarting Shib service, etc. Then our out-of-town consultant suddenly told me it was consistently working for him on multiple browsers! But it continued to fail for me at the office. When I got home, I tried again, and this time it worked consistently. At the office it seemed not to work for me but did for some people.

Based on comparing the headers on working and non-working servers, we made a few more changes to shibboleth2.xml - mainly adding explicitly stated 443 port to the hostname

<Site id="1" name="proxyserver.com:443/content" scheme="https" port="443"/>

<Host name="proxyserver.com:443/content" authType="shibboleth" requireSession="false" scheme="https" port="443">
<Path name="ct" authType="shibboleth" requireSession="true"/>

Then I left for vacation, assuming it was still unstable, since it did not appear to work at all for my office computer .

When I got back the consultant was telling me that it was totally working. I had everyone on my team test both at the office and at home and it just works. At that point I halted the reverse proxy change request.


My two questions are:

1. When I got back to the office after vacation it STILL did not appear to work on my office computer! Only mine. Everyone else was working. I cleared all cookies from the browser, cleared the cache, restarted the browser, but it still redirected to web root. Then I restarted the computer and it started working consistently for me. What caused this if not a cookie? We are starting to think that perhaps forms auto-fill was auto-filling an old incorrect signed statement into the form that is POSTed in the background back to the SP.

I need to understand because this sort of false result can make it very difficult to troubleshoot.


2. Is there any reason I should still go forward with changing the reverse proxy mapping that I halted?
https://proxyserver.com/content/ ==> https://contentserver.com/content/


Thanks!




______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org
Phone [phone]

Post by Cantor, Scott

Post by Meiselman, Ellen
______________________________
Line 112. https://reverseproxy.com/content/Shibboleth.sso/SAML2/POST

For that to work, you'd have to have a handlerURL on the SP of
/content/Shibboleth.sso. In addition, the request to that server had
better be for /content/Shibboleth.sso/SAML2/POST. If it's not, the SP
won't handle it, and even if it did, it wouldn't match the URL that's
embedded inside the message from the IdP and will be rejected. If it's
/content on the front-end, it has to be /content on the back-end.

Post by Meiselman, Ellen
Note that a new _shibstate cookie is set at this spot with a new number
appended to its name. Its value is now /content//Shibboleth.sso/SAML/POST

That's because your SP doesn't believe that that is the location of its
handlers, *and* it is being told to protect all URLs with /content in
them. Combine the two and it sees that as a protected resource and issues
a request for a login. You don't need to see a loop to see that, just
access /content/Shibboleth.sso/Foo and when that redirects to the IdP, you
know you have an unworkable system.
The only way the SP will exempt URLs that match protection rules is if
they're for its handlers, and it determines that using the handlerURL
setting.

Post by Meiselman, Ellen
* Note that the path (/secure) to the requested resource is set by the
Shibboleth SP and hence is specific to the protected resource on the web
server. This mandates that the proxy either proxies the resource with the
exact same path (/secure to /secure), or that the proxy is able to
rewrite HTTP resonse headers (e.g. the ones containing
the relayState) before returning results to the client.
https://wiki.shibboleth.net/confluence/display/SHIB2/SPReverseProxy
We're mapping /content to / (webroot) So that could be one problem at least.

It is. You can't do that. The suggestion above won't help either. That
might work if the content URLs were the ones with the mismatched paths,
but you can't do that with the handlers. The message from the IdP will
contain the URL to the proxy, and if that includes a path, the same path
has to be in the request to the back-end server as seen by the SP or it's
going to reject the message.
You can rewrite cookies and RelayState, but you can't rewrite a signed
message.
So full stop, you can't do it.
-- Scott
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[Cantor, Scott]

Post by Meiselman, Ellen
We added a virtual directory called "/content/" to the SP content server
which pointed to a real directory at the root of the server then added
another directory inside it "ct" that points to the resources directory.

Right, I mean, the point is that as long as it's all virtual anyway, there
is no reason to create different paths on the proxy as on the resource
server.

Post by Meiselman, Ellen
https://proxyserver.com/content/ct/resource.htm
it would redirect to the web root
https://proxyserver.com/content/

Redirect how? When? After what step?

Post by Meiselman, Ellen
Based on comparing the headers on working and non-working servers, we
made a few more changes to shibboleth2.xml - mainly adding explicitly
stated 443 port to the hostname
<Site id="1" name="proxyserver.com:443/content" scheme="https"
port="443"/>
<Host name="proxyserver.com:443/content" authType="shibboleth"
requireSession="false" scheme="https" port="443">
<Path name="ct" authType="shibboleth"
requireSession="true"/>

No. Those "name" attributes are hostnames only. The fields are broken out
so that the code doesn't have to parse URL syntax to find out the intent.
You're fooling the code in some weird way that is not supported. Even if
it worked, I couldn't tell you why and I guarantee it won't keep working.

Post by Meiselman, Ellen
1. When I got back to the office after vacation it STILL did not appear
to work on my office computer! Only mine. Everyone else was working. I
cleared all cookies from the browser, cleared the cache, restarted the
browser, but it still redirected to web root. Then I restarted the
computer and it started working consistently for me. What caused this if
not a cookie? We are starting to think that perhaps forms auto-fill was
auto-filling an old incorrect signed statement into the form that is
POSTed in the background back to the SP.

I have never heard of any auto-filling of the SAML forms.

Post by Meiselman, Ellen
2. Is there any reason I should still go forward with changing the
reverse proxy mapping that I halted?
https://proxyserver.com/content/ ==> https://contentserver.com/content/

You can listen to me or not, I wrote the software, and I'm telling you
that you cannot put ports or schmes into the name field of a Site or Host
element. That is not a correct configuration.

-- Scott

[Cantor, Scott]

Post by Cantor, Scott

Post by Meiselman, Ellen
2. Is there any reason I should still go forward with changing the
reverse proxy mapping that I halted?
https://proxyserver.com/content/ ==> https://contentserver.com/content/

You can listen to me or not, I wrote the software, and I'm telling you
that you cannot put ports or schmes into the name field of a Site or Host
element. That is not a correct configuration.

What I think it's doing is fooling the code into accidentally behaving
somewhat like a rewrite engine and producing URLs that are superficially
doing the right things. I suspect that if you never had to change the
content scenario here, and never use the RequestMap for anything more than
this, it might work, but I can't guarantee that some future change doesn't
break it. If I add something that needs to use those configuration
elements, I'm counting on them carrying the hostname alone.

-- Scott

[Meiselman, Ellen]

OK. Got it.

I have ordered a second VM and will set that one up as recommended.

Thanks so much for your help!

Ellen


______________________________
Ellen Meiselman
University of Michigan Health System
MLearning
NCRC
2800 Plymouth Rd.
Building 200, Rm 207
Ann Arbor, MI 48109-2800
E-Mail: emeiselm-63aXycvo3TyHXe+***@public.gmane.org
Phone [phone]

Post by Cantor, Scott

Post by Cantor, Scott

Post by Meiselman, Ellen
2. Is there any reason I should still go forward with changing the
reverse proxy mapping that I halted?
https://proxyserver.com/content/ ==> https://contentserver.com/content/

You can listen to me or not, I wrote the software, and I'm telling you
that you cannot put ports or schmes into the name field of a Site or Host
element. That is not a correct configuration.

What I think it's doing is fooling the code into accidentally behaving
somewhat like a rewrite engine and producing URLs that are superficially
doing the right things. I suspect that if you never had to change the
content scenario here, and never use the RequestMap for anything more than
this, it might work, but I can't guarantee that some future change doesn't
break it. If I add something that needs to use those configuration
elements, I'm counting on them carrying the hostname alone.
-- Scott
--

**********************************************************
Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues