April 15, 2020
By Chris Hyzer, University of Pennsylvania
There have been exciting advancements in the Grouper project recently, most notably with the release of Grouper v.2.5. Grouper is the enterprise group and access management system that is part of the InCommon Trusted Access Platform.
If you haven’t yet deployed Grouper, we invite you to give Grouper 2.5 a spin. With the new container strategy, deploying Grouper has become simpler and more straightforward than ever before. A handful of institutions have already upgraded to v2.5 and they needed little to no help. The new container greased the skids.
For current Grouper deployers, it seems that there are generally two strategies of which Grouper version to use. The conservative strategy is to run Grouper one version back from latest, and you have a stable static environment. You might need an infrequent change if there is an important bug or security fix. If you are on Grouper v2.3 right now, then if you follow this conservative strategy, you should upgrade to the latest v2.4. In Grouper v2.4, you can use a container or not, though using a container is recommended. If you look at the Grouper Roadmap for items completed in Grouper v2.4, you will find at least two dozen enhancements. Grouper v2.4 will now not see any new enhancements as the Grouper development team focuses on v2.5 and v2.6.
If you have a somewhat recent version of Grouper v2.4, and you enjoy being able to benefit from the latest Grouper features and improvements, then Grouper v2.5 is calling your name. Since we put most new development in the current version (used to be patches, now it’s new container versions), the leap from the most recent v2.4 to v2.5 is not a large one, especially if you currently run on a container.
In the past, Grouper used to be constrained since we did not want to make database structure changes (DDL) in a build number. But now we have an automatic (optional) way to do that so we have more flexibility in adding non-core enhancements. Don’t worry, these changes are backward compatible if your web service is a different version than your daemon or if you want to roll back a container.
What’s a Container?
Everyone keeps saying “container,” what does that mean? The container is the Grouper code, application server (TomEE), web server (Apache), Java (exact supported and tested vendor and version), Shibboleth, operating system, etc. so everything is one big black box. It connects to the external database, it sends logs somewhere, and you customize it with configurations and overlays, but everything is all standard, structured and organized.
With the container approach, you can’t install a bad patch, you can’t botch an install from the installer, you can’t upgrade wrong. There’s a LOT of consistency and a reduction of unknown unknowns. Upgrades (new container) are a lot lower risk and easier to do, with instantaneous ability to roll back. It’s a great new architecture. People still argue about how to do it right, and we have made a lot of compromises among the camps. but in general it’s a very big improvement. We do not mean you need to run in a Docker runtime (though our documentation on quick start might lead you to believe that). Institutions run Grouper in various runtimes including Docker, AWS Elastic Container Service, Kubernetes, Azure, etc. “Container” does not equal “Docker”.
Kind of fitting, the container is the elephant in the room. In Grouper v2.4 we built a container around how the Grouper Installer built Grouper. This was fine, and it worked and was a welcome change from running disparate versions of Tomcat, but it wasn’t ideal. There were four copies of configuration files, two app servers, and it wasn’t exactly clear how to toggle certain services or customize. Granted, we had a way to simplify some of those things, but it was still confusing and error prone. In Grouper v2.5 we started over, and completely revamped the structure of the container with the mindset that the container is central and required (will get to that in next paragraph).
Now there is one copy of each configuration file, one application server (TomEE which is a muscly Tomcat), and there are configuration options to provide all the flexibility needed to run Grouper however you want (Apache y/n, Shibboleth y/n, memory settings in one place, etc). Further, things that aren’t conducive to containers (e.g. tomcat-users.xml, editing a web.xml merged from ui/ws/scim) are now removed from the picture. Converting a container configuration (e.g. Dockerfile) from Grouper v2.4 is not expected to be a heavy lift (wire things to a slightly different location). Speaking of which, there is a lot more documentation about the container to help you get started or upgrade. And the Grouper Installer will install the container and save a file of commands for newbies.
Maturity Levels
As a community we discussed the various ways to run containers (dubbed “maturity levels”), and documented how to evolve with containers. The Grouper v2.5 container can be up and running in five minutes with the Grouper Installer (if you have a server and a database already), and the container can run as is without making a sub-container (maturity level 0). Do we recommend this as best practice for a long-term production container environment? No.
The container maturity levels will guide you to the ideal orchestrated container environment. In maturity level 1 you create your own sub-container (e.g. if you are using Docker then provide a Dockerfile). Maturity level 2 suggests that you make your container independent of the server it runs on by running a secret manager and a network logging service. Maturity level 3 automates scaling and deployment with load balancing, monitoring, and container orchestration. For more on container maturity levels, see this wiki page.
Grouper no longer ships in a tarball. The Grouper Installer does not install Grouper from source. Patches are no longer released and no patch class files are in the Java classpath. We release a container from Docker Hub and have a couple jars (client and installer) from Maven central. All jars in the container are from Maven central so we have solved a consistency problem. We have evolved.
We did not make this edict of requiring the container solely to drastically reduce Grouper support time to troubleshoot heterogeneous deployments, failed patches, and frustrating upgrades. It also reduces the amount and effort of FTEs required to deploy and support Grouper at your institution. It’s a leap of faith but it is also a win-win.
Hopefully with the “Maturity Level 0” documentation and features, institutions who aren’t yet ready for containers, or who “aren’t allowed to run containers” will still be able to deploy and run Grouper v2.5 in production. And when they get their container practice up and running, they can easily lift and shift from the “Maturity Level 0 or 1” over to their shiny new orchestrated container environment (we don’t care what stack they choose).
You might have noticed that I generally prioritize access management features of Grouper over the behind-the-scenes tech stuff. We just removed Apache “ant” in 2020. However, I’m very excited about containers. If you only knew how much time it took to make/test a patch in Grouper v2.4. Ten times that of cutting/testing a new Grouper v2.5 container. Our new CI/CD (continuous integration/continuous deployment) is great and automatic testing will reduce bugs.
Access Management Features
Ok, let’s get to new Access Management features of v2.5. Grouper is about groups. The feature set of Grouper aligns with RBAC (Role Based Access Control). One gap we had from full RBAC compliance is the ability to enable/disable a group (we could disable a membership but not a group). In Grouper v2.5 it can be done. Set up a group with an enabled date in the future. Everyone who asked “Where are the teeth of attestation? Should the group temporarily disappear if not reviewed on time?” In the medium future that will be possible.
There are new web service operations (query the audits, query point in time state of access, improved paging to efficiently and reliably retrieve large datasets). Wondering why performance of daemons has issues? There is a Gantt chart (a la visualization) of deamon jobs to help troubleshoot that. The UI has a “Custom UI” where you can target access decisions and help end users and managers troubleshoot access
Looking to the Future
What does the future of Grouper hold? As our completed roadmap items grow, as do our requests for new things. The current direction is to focus on provisioning (making it more efficient and reliable, easy to configure in the UI, and easy to troubleshoot in the UI). We’ve already started making good progress. Maybe that motivates you to get to Grouper v2.5? After that we would like to make more administrative wizards (e.g. to help in the configuration of subjects sources). Let us know if you have ideas for Grouper or suggestions for priorities.
To learn more about the Grouper 2.5 release, to download the software and release notes, for upgrade instructions, and a link to a Grouper demo server, please visit: https://spaces.internet2.edu/display/Grouper/Grouper+Downloads
For an introduction to Grouper, please see the Grouper Deployment Guide (revamped December 2019 and in wiki form).
Online Grouper Training in June 2020
There is a virtual training “Grouper School” June 2-3, 2020 for access enthusiasts of all levels. Please encourage any colleague to enroll who you feel could improve their access management practices. You are invited to visit the Grouper website, wiki, and join slack and the email lists.
We encourage you to share your experiences with Grouper containers on a Community Contribution wiki page. Shout out to University of Maryland College Park (scroll down on their Grouper contribution page) for setting an example.
Thanks,
Chris Hyzer on behalf of the Grouper Team