Service Design Review Checklist for SOA Governance

One thing that seems to come up with some frequency in SOA circles is creating a checklist that would help make the development team, the architecture group, SOA Governance board, and the business customer feel confident about rolling out a service--something that would act as a guide so that developers know they're making something appropriate and useful, to help encourage good SOA design principles, and to make sure that the business actually realizes the benefits of SOA (instead of just moving the problem).

So I thought I'd put this checklist together and hopefully save you some effort. This is probably something I should included in Java SOA Cookbook, but here it is now.

This checklist for Service Implementation includes 90 questions for ensuring that a proposed candidate service will make a good addition to your catalog, and has a good chance of being successful after it's gone live. 

Here’s how this is intended to be used in the context of SOA Governance:

1. Introduce this Review Checklist to the Application Architect or development team who will design and realize the service—before they start working on it. If they know this test will be coming at the end, they will account for these things. Just as with Test-Driven Development where you write the code that makes the test pass, this is Test-Driven Service Design—create the service that will pass the test below. No Programmer Left Behind!
2. As the service is in development, the team should consult back with this checklist to make sure nothing is being forgotten. Of course, not everything will apply for every service.
3. When the service is nearing the end of the development cycle, the Application Architect or whoever is the lead developer of the service should present her service before the SOA Governance Board, so that they can be responsible for introducing it to production deployment and the service catalog. If too many items have been left unconsidered by the development team, don't let the service be promoted.

 Tailor this list for your own organizational needs, of course. Here's the checklist:

Background

1. What business assumptions have been made? What risks do they pose?
2. What system assumptions have been made? What risks do they pose?
3. What specifically does this service do to address the following:
1. Decrease costs
2. Increase revenue
3. Promote agility
4. Promote productivity
4. What is the business governance domain for this service?

 

Service Design

1. What is the category of this service?
1. Stateful Business Process (Employee Onboarding, Return Merchandise)
2. Business Entity (nouns such as Employee, Customer)
3. Business Functions (verbs for atomic actions in a process). May also be Event Handlers
4. Utility (perform an application-agnostic function such as Email)
5. Security Service (handle identity, authorization, privacy)
2. How has the design iterated or evolved? Did you start by considering the consumer view?
3. Describe how the design used a “middle out” approach.
4. Where are the tightest couplings with other services, other systems, etc?
5. How do you anticipate this service being reused? In what systems? By what kinds of consumers? How can modifications be minimized?
6. What patterns from the Service Design Patterns catalog have been employed? (If you don't have your own internally approved SOA Patterns catalog, start out using Thomas Erl's excellent http://soapatterns.org).
7. Have you followed relevant organizational implementation standards (Java coding conventions, etc)?
8. How have you accounted for internationalization? How will your service support localization (eg, return different data based on geographic location, formatting concerns for currency, language, and other items)?
9. What transports does your service support (SOAP, JMS, HTTP, etc)? Why were those selected?
10. Does the service make use of or allow for user preferences (eg, number of results returned, etc).
11. What are the basic Message Exchange Patterns used for this service?
12. How does the design support an event-driven approach?

 

Service Operations

1. Does the service support purely stateless connections (unless it is a business process service)?
2. Do service operation definitions support typical variations in the domain?
3. Have you avoided any messages, operations, or logic that are consumer-specific?
4. Are all operations capable of being executed independently, without necessarily relying on any previous invocation of another operation?
5. Are all data access operations idempotent?
6. Does the service offer a variety of operations for retrieving minimal, most common, and full data sets?
7. Does the service use only standard logging facilities and a log rotation strategy?

 

Coupling

1. Name all external systems called by this service.
2. Does this service wrap an existing legacy system or database? Could that system be entirely replaced by a newer or different implementation without affecting consumers?
3. How does the service capture inputs and outputs as business documents? How does your level of abstraction avoid RPC?
4. Does this service not directly invoke another service, but instead pushes such invocations up to an orchestration?

 

Business Processes/Rules

1. Have any specific business processes have been identified that can use this service in automation?
2. What business rules have been identified that can be extracted to a business rules management system or external rules engine?
3. Does the service reference any business rules that may feature thresholds or other items that could be configured by a business user, or are they baked into code?
4. What KPIs have been identified for the service?

 

Function

1. How does the design fulfill the functional requirements of the service?
2. How have the boundary cases been considered?

 

Data

1. Describe how this service accesses data, what data it accesses, and where.
2. Are transactions required? How does the design handle transactions? Has compensation been considered as an alternative?
3. Describe how this service fully encapsulates its data. If it cannot at this point, what is the transition plan for doing so?
4. Describe how this service uses the Canonical Data Model.
5. How does the service perform validation on incoming data? How does the service respond to invalid inbound data?
6. How does the service account for data quality?
7. Have you externalized strings?

 

Service Contract

1. Have relevant current and potential future consumers been consulted on the service contract?
2. Will this service use virtualization? Why or why not?
3. How does the service support the principles of Service Orientation?
1. Loose coupling
2. Abstraction
3. Composability
4. Standard Contract
5. Autonomy
6. Statelessness
7. Reusability
4. Does the service contract need to vary for external business consumers as opposed to internal application consumers? How so?

 

Errors

1. Does the service use only standard message return codes and user-friendly descriptions?
2. What checked exceptions (faults) does the service offer? Under what circumstances can they be generated?
3. What runtime exceptions are likely to be generated from the service? What result do you expect in consumers receiving runtime exceptions? How can this be mitigated?

 

Performance

1. What is the measured latency of service response in testing?
2. What SLAs have been defined for this service? What mechanisms are in place to prevent SLA violations? What mechanisms are in place to report SLA violations?
3. What steps in an orchestration can you design to be executed in parallel and joined later?
4. How does the design encourage asynchronous invocation?
5. Are the operations designed at various levels of appropriate granularity so that they are not prone to network chattiness and do not return data clients are not likely to need?
6. How does the design delineate between operations that must be performed quickly and operations that are long-running?
7. Can the service be scaled be adding more nodes running the service? What might prevent this?
8. What is your caching strategy behind the service implementation? Can known consumers easily cache data in front of the service? How will this be managed (eviction policy, invalidation, etc)?
9. If your messages use binary data, do they employ MTOM?
10. Does your design allow for clients to select variations on an operation based on their context? For example, do you offer both doXandWait(m) : Response and a doXLater(m) : Void operation options?

 

Security

1. Does the service require authentication? Authorization? How are these implemented?
2. What other regulatory constraints (PCI, Sarbanes-Oxley, etc) might affect this service contract or deployment? How have those been directly accounted for in the design?
3. What are any additional security requirements for this service? How are they fulfilled?
4. Does the service allow for auditing? How is that implemented?
5. Are logs free from PCI or PII information?

 

Quality Assurance

1. How many unit tests are available for this service?
2. Are all unit tests independently executable (ie, not dependent on the successful run of any prior test)?
3. Were test cases created for every user function? Did the tests use a variety of data inputs (valid, invalid, null, many different combinations of length and character)?
4. Were test cases created for all exception conditions?
5. Were test cases created around a generated client (in Java or .NET)?
6. Are the unit tests in version control, and versioned in clear correspondence with the service so that the environment can be entirely reproduced?
7. What is the test coverage percentage for this service?
8. If this is a new version of an existing service, have you tested directly for backwards compatibility issues?
9. What functional tests were written if a consumer is available?
10. How was the service load tested? What metrics were recorded?
11. If the service uses asynchronous or fire-and-forget operations, were these tested by subscription?

 

Lifecycle

1. Are the service and attendant schemas appropriately versioned?
2. What can be retired or sunsetted after this service is deployed?
3. Have you externalized configurable data?

 

Availability & Support

1. Will the service be highly-available? What are the availability requirements? How will these be met? What is the business impact if the service is down for 1 minute? 5 minutes? 30 minutes? 1 hour? 4hours?
2. How will availability be measured?
3. How will the production support team receive messages or alerts regarding the current state or health of the service?
4. How will runtime issues with the service be addressed organizationally? Has an on-call schedule been established?
5. What visibility does the service directly offer (eg, in the form of JMX) to different support teams?
6. Does the service require planned downtime for maintenance? How much time, and how often? What is expected to be performed during this down time?
7. How have you involved the infrastructure team in the creation and design of this service?
8. What is the plan for future maintenance of the service after it is successfully deployed?

 

Deployment

1. What problems is service deployment likely to pose?
2. Will the service be load balanced?

 

Documentation

1. Have you captured the design in the Service Template?
2. Has the service been captured
3. Have you followed relevant standards for code-level documentation?
4. Have all test execution results been recorded (eg, in the Maven site)?

The blog ArtOfSoftwareReuse.com has a very good checklist too, and I've incorporated some of the excellent work done by Vijay there at http://bit.ly/3UJaNk. That checklist has fewer than 50 items; mine is approximately twice the length. Of course, it's expected that you'll use only what makes sense for your organization. Good luck!