The performance of Shibboleth

[Shen Hongzhou]

Hi,all
Anybody have the performance data of Shibboleth?
I mean throughput, number of requests per second etc.
Thanks!

Shen

[Peter Schober]

Post by Shen Hongzhou
Anybody have the performance data of Shibboleth?
I mean throughput, number of requests per second etc.

There's a topic on load testing for IdP in the wiki
https://spaces.internet2.edu/display/SHIB2/IdPProdLoadTest
Maybe someone mentioned some of those numbers in the archives,
but it hasn't been recorded anywhere else, AFAIK.
-peter

[Shen Hongzhou]

Thanks Peter,But unfortunately, I didn't find anything useful.

Shen

Post by Peter Schober

Post by Shen Hongzhou
Anybody have the performance data of Shibboleth?
I mean throughput, number of requests per second etc.

There's a topic on load testing for IdP in the wiki
https://spaces.internet2.edu/display/SHIB2/IdPProdLoadTest
Maybe someone mentioned some of those numbers in the archives,
but it hasn't been recorded anywhere else, AFAIK.
-peter

[Peter Schober]

Post by Shen Hongzhou
Thanks Peter,But unfortunately, I didn't find anything useful.

Well, then you have your answer ("No").
-peter

[Chad La Joie]

Because there is no way to answer the question you asked. Shibboleth
runs as well as it runs on the hardware you install it on, the JVM you
use, the container you use, the tuning options for the OS, JVM, and
container you put in place, your specific IdP configuration, etc. The
only meaningful numbers you'll ever get are retrieved by installing the
software and running the provided load tests against it.

But since you want numbers. Shibboleth will easily handle 1 million
requests simultaneously with less then 100ms response times given
sufficient hardware resources and network topology.

Post by Shen Hongzhou
Thanks Peter,But unfortunately, I didn't find anything useful.
Shen

Post by Peter Schober

Post by Shen Hongzhou
Anybody have the performance data of Shibboleth?
I mean throughput, number of requests per second etc.

There's a topic on load testing for IdP in the wiki
https://spaces.internet2.edu/display/SHIB2/IdPProdLoadTest
Maybe someone mentioned some of those numbers in the archives,
but it hasn't been recorded anywhere else, AFAIK.
-peter

[HaiJun Deng]

Post by Chad La Joie
Shibboleth will easily handle 1 million
requests simultaneously with less then 100ms response times given
sufficient hardware resources and network topology.

At
https://wiki.shibboleth.net/confluence/display/SHIB2/IdPProdLoadTestResults
<https://wiki.shibboleth.net/confluence/display/SHIB2/IdPProdLoadTestResults>
, it said " A single c1.xlarge instance running in the same fashion can
achieve 300 transactions per second and is not susceptible to freezes or
crashing during garbage collection.", it seems that the performance of
shibboleth is not good, is it?



--
View this message in context: http://shibboleth.1660669.n2.nabble.com/The-performance-of-Shibboleth-tp3595144p7589995.html
Sent from the Shibboleth - Users mailing list archive at Nabble.com.
--
To unsubscribe from this list send an email to users-unsubscribe-***@public.gmane.org

[Kevin P. Foote]

Post by HaiJun Deng

Post by Chad La Joie
Shibboleth will easily handle 1 million
requests simultaneously with less then 100ms response times given
sufficient hardware resources and network topology.

At
https://wiki.shibboleth.net/confluence/display/SHIB2/IdPProdLoadTestResults
<https://wiki.shibboleth.net/confluence/display/SHIB2/IdPProdLoadTestResults>
, it said " A single c1.xlarge instance running in the same fashion can
achieve 300 transactions per second and is not susceptible to freezes or
crashing during garbage collection.", it seems that the performance of
shibboleth is not good, is it?

I do not belive you are thinking about this correctly.

Your use case puts 5k tps .. this is on the application. The SP is the
piece directly involved with the protected application side of things.

Also the test listed above was one example, not that there are many
more posted. However, what your IdP does tps wise is not reflective of
what your SP is doing nor what your actual protected app is doing.
People quite routinely have very large scale deployments of both the
Shib-IdP and SP in enterprise settings.

I would tend to believe the statement of Chads regardless of what TPS you
are hoping to achive..

Generally speaking the IdP does exactly what it is supposed to do in a
very efficient manor. Your time at IdP will vary greatly depending on
what back end systems your IdP relies on and how fast the access to
those resources is.

------
thanks
kevin.foote
--
To unsubscribe from this list send an email to users-unsubscribe-***@public.gmane.org

[HaiJun Deng]

In that post, it says "A single c1.medium instance running the IdP in Jetty
without Terracotta and a million user records and a standard set of
attributes in an LDAP directory with authentication randomly choosing one of
them can handle roughly 65 transactions per second."
it seems that the SP does nothing except sending the authentication request.

Anyway, I am here not to belittle the performance of the shibboleth, i just
wonder what the usual throughput the IdP can achieve without thinking of the
effection of SP and APP.



--
View this message in context: http://shibboleth.1660669.n2.nabble.com/The-performance-of-Shibboleth-tp3595144p7590005.html
Sent from the Shibboleth - Users mailing list archive at Nabble.com.
--
To unsubscribe from this list send an email to users-unsubscribe-***@public.gmane.org

[Cantor, Scott]

Post by HaiJun Deng
In that post, it says "A single c1.medium instance running the IdP in Jetty
without Terracotta and a million user records and a standard set of
attributes in an LDAP directory with authentication randomly choosing one of
them can handle roughly 65 transactions per second."
it seems that the SP does nothing except sending the authentication request.

The SP does a lot of things, but the IdP can be load tested with no real
involvement from an SP. Regardless, the typical way people approach their
requirements for the IdP is crazy. They will throw out numbers like 500
logins per second with no earthly clue how many actual logins per second
they would have. Usually that number is 100 or more times lower than it's
claimed to be even at peak.

Post by HaiJun Deng
Anyway, I am here not to belittle the performance of the shibboleth, i just
wonder what the usual throughput the IdP can achieve without thinking of the
effection of SP and APP.

And that is an unanswerable question. Software runs as fast as the
hardware you put it on constrained by the other back end components it has
to rely on.

An IdP is either I/O bound due to database or directory issues, or CPU
bound from cryptography. The latter scales linearly a long way, and if
configured statelessly, it scales linearly across essentially any amount
of hardware. I don't even think it's ever been tested with a PKCS12 module
to do the signing, which might boost performance by an order of magnitude.

-- Scott


--
To unsubscribe from this list send an email to users-unsubscribe-***@public.gmane.org

[Kevin P. Foote]

Post by HaiJun Deng
In that post, it says "A single c1.medium instance running the IdP in Jetty
without Terracotta and a million user records and a standard set of
attributes in an LDAP directory with authentication randomly choosing one of
them can handle roughly 65 transactions per second."
it seems that the SP does nothing except sending the authentication request.
Anyway, I am here not to belittle the performance of the shibboleth, i just
wonder what the usual throughput the IdP can achieve without thinking of the
effection of SP and APP.

And as you have seen .. those are the only figures that are posted. I do
believe that your best bet is to spin up your 5k+ tps App with a
properly tuned web server and SP combo and see what you get.

As I said before I'd tend to believe the statement(s) by Chad regarding
IdP performance.


------
thanks
kevin.foote

--
To unsubscribe from this list send an email to users-unsubscribe-***@public.gmane.org