This is a public Blog  publicRSS

Entry

    Chris Warner
    Goldilocks and the Three Testing Scenarios
    Entry posted October 5, 2016 by Chris WarnerWhiz, tagged Best Practices, Product / Product Release 
    576 Views, 13 Comments
    Title:
    Goldilocks and the Three Testing Scenarios
    Entry:

    Goldilocks and the Three Testing Scenarios

    Sitting in her plush, window-view corner office in the Log Cabin Bedding Enterprises corporate building, CSR Manager Goldilocks was reviewing the latest message from Oracle regarding the upgrade project for her Oracle Service Cloud site. Thinking aloud to herself (which she often did), “Well, Oracle Upgrade Engineering has finished their work for the upgrade project and have handed-off the Upgrade site to us for User Acceptance Testing (UAT). How should I get this done to make sure all of our customizations and critical functionality perform the same with this new version of the product?”

    Thinking to herself, Goldilocks tried to work through different testing approaches to complete UAT for the upgrade project. “Testing every piece of functionality that we use in the product will require an extraordinary amount of resources and I know our CEO, Nathanial “Pappa” Bear, would not be happy if I spent a lot of resources on an upgrade project. That sounds like too much testing.”

    “Or, I could just run through the Upgrade site myself, or maybe even have one of my CSR reps do it real quick. However, if we miss something in our testing that causes a problem after the upgrade to the new version, then our VP of Marketing, Chris “Baby” Bear, will be really upset and I’ll be in meetings and writing Root Cause Analyses’ for weeks. That sounds like too little testing.”

    “Maybe, just maybe, I can come up with a way that will test all of our customizations and business critical functionality but not go too far or involve a large resource cost. A middle-of-the-road method that will still determine if there are any issues with the new version. This would keep management happy and give us access to all the newest functionality available in the latest release from Oracle Service Cloud, making our COO, Erica “Mamma” Bear, super happy. The Browser User Interface, Chat functionality updates, Social Monitoring, etc., are exactly what she has been looking for in the product. That sounds just right, I think.”

    Continuing her thinking out loud (lucky for us readers), Goldilocks envisioned what her preparations needed to be to make her ‘just right’ testing plan successful. “Let’s see, my upgrade project manager from Oracle provided a link to one of their knowledgebase answers with some guidance for upgrade testing…Answer ID 8643, I think. That’s it! Let’s see, I can get started with my testing plan. Setting up Test Servers with our IT staff to test CRON jobs or data transfers, got it. Making sure our workstations are compatible with the latest version requirements. I’ll have Dave do that. Documenting all of our critical business workflows will be his task. That’s a good idea. That way, I can use anyone in the CSR group to test, since they will have it all laid out in front of them. Testing first in Production and then in the Upgrade site and noting any differences…”

    And on she planned (out loud), eventually getting her plan just right. “This will be perfect. We can skip over the basic product functionality and concentrate on those areas that are critical to our business. That will ensure that we have tested the areas that will impact our business, should there be any changes in the product’s behavior. And, we can keep the resource requirements to a minimum.”

    The next week, after UAT was completed and the upgrade cutover finished, Goldilocks was reviewing the upgraded site and completing a few last-minute post-cutover validations. Thinking to herself (yes, out loud again), “That went great!  And, I have read some Community posts about the Auto Upgrade Program and how it could keep my site upgraded regularly, with every quarterly release of the Service Cloud product.  I wonder what it would take to get enrolled into that Program?” And, as often happens in fairy tales, a new email arrives from Oracle, telling her that her site is now eligible for the Auto Upgrade Program.  Thinking, “This will be awesome! I don’t have to perform UAT testing anymore – all my customizations are built using Oracle’s Managed Frameworks however, I can still test if I wanted to. A win-win for the company. I’m going to enroll!” 

    And everyone in the CSR group lived happily ever after at Log Cabin Bedding Enterprises.  

    Comment

     

    • JustRhianna

      Very entertaining and useful

    • Colin Campbell

      Nice post...

      I am looking forward to the 3 Little Pigs and the Big Bad [Something] 

    • Jess Campbell
      We have a checklist. It was a project of mine last year.
    • Robert Pozderec

      Checklist that covers all major components is the way to go. 

    • Tasha Sylvester

      Important information, thank you for posting.  The prep for an Upgrade is critical as well as defining re-usable material to allow the application team to run with their portions to test and document everything is operational as intended.

    • Chandini Davis

      Who doesn't love a good story and a good checklist?

       

      • Read post - Done
      • Comment on Post - Done
      • Rehab your testing checklist - Pending
      • Make a list of lists - Done
    • Bryan Arndorfer

      What I enjoy about this is the struggles of testing.  Too much and its a waste, too little and you pay for it with post upgrade work.  That goldilocks scenario is exactly what you want to achieve!  And in my experience has been what we get to after a few tests of the testing plan and finding out when we did too much or too little.

    • Kurt Helfrich

      I think that the custom tasks on the checklist are an awesome feature.  We have a lot of customizations, so the custom tasks and folders help us manage testing of our work.

    • Shannon Greene

      I have a checklist to run thru on my own (basic stuff), and then I let the users run thru a few more in-depth scenarios that they would do in their normal workflows. At that point, we have hit all our major functionality and customizations.

      We only tend to have issues with the integration service we use to allow Omni-channel functionality. 

    • Julia Neary

      Agree with the other posters -  a checklist is the way to go. We spend most of our time testing our customizations and add-ins.

    • Franky Weber Faust

      Cool and very funny.

    • Marc Grant

      We have a Jira item with sub-tasks for each area we test and the go-live cutover tasks. Each sub-task can be assigned to the relevant person and due dates can be added if necessary. We simply clone that for each upgrade, considering our lessons learned from the previous upgrade and adding/editing tasks as required..

      I'm not saying we're perfect, but the team are getting pretty good, although the challenge, as always is getting all integration points set up for end-to-end testing.

      We're a few versions behind at the moment, so want to get caught up soon, and see how much closer we are to the AUP this time!

    • Chris van Es

      Agree. We use checklist as well since 2007 and are now on the auto upgrade program. The upgrade manager from Oracle is a great help.