Project

General

Profile

Feature #1

create an example fully defined test plan (including the related test sets, tests, screens, files...)

Added by Greg Shah about 12 years ago. Updated over 11 years ago.

Status:
Closed
Priority:
Normal
Start date:
Due date:
% Done:

0%

billable:
No
vendor_id:
GCD

hello_world.xml Magnifier (1 KB) Eugenie Lyzenko, 08/24/2012 06:43 PM

hello.xml Magnifier (981 Bytes) Eugenie Lyzenko, 08/24/2012 06:43 PM

get_sys_parms.xml Magnifier (595 Bytes) Eugenie Lyzenko, 08/24/2012 06:43 PM

History

#1 Updated by Greg Shah about 12 years ago

The objective is to create something that can be used as a reference or as samples to help users learn to encode a testing baseline.

This would be very useful, however it will have to be something that is essentially non-functional since otherwise a fully working application will have to be provided in order to test the example test plan.

#2 Updated by Greg Shah over 11 years ago

  • Assignee set to Eugenie Lyzenko
  • Status changed from New to WIP

If we make some assumptions about the "application" being tested, then we can make a functioning test plan that can be tested.

For example, let's assume that the application is running on Linux and anything that can be run in a bash shell is possible. Pick programs that are very common in Linux so that a running Linux environment is enough to run these example tests.

You can write shell scripts to do specific things like read input and generate output. You can use things like "cat" to "output" text files to the screen. You can use "read" in shell scripts to get input. You can use "sleep" with a random number of seconds to simulate long running processes. Using simple techniques like this, you can simulate a real "application", for which you can write tests.

Make a survey of the most common patterns used in the Majic baselines. Then try to duplicate these scenarios using the Linux command line tools noted above. Write tests to fully automate the testing of these scenarios.

The objective is to make these tests useful for educational purposes. This means that you must document each test carefully.

Make sure to simulate the more complex issues too, such as usage of run-code, semaphores/threading, multiple instances of the same test being run, CTRL-C processing...

The purpose is to provide an example baseline that people can learn from. It must really use the same techniques as we have found to be important for Majic.

#3 Updated by Eugenie Lyzenko over 11 years ago

The question:

OK. Another word I need to make some "application neutral" test cases with harness features we used in TIMCO baseline set. To be able to run on every "clean" Linux box. Every test should have detailed description of what it does and how it is implemented. After the person will learn these test he should be able to write the new tests for something like MAJIC using all power of the harness facility. Correct?

#4 Updated by Greg Shah over 11 years ago

Yes, exactly right!

#5 Updated by Eugenie Lyzenko over 11 years ago

The features planned to be covered in samples:

General structure of the configuration files
Delays
Mutexes
Event semaphores
Multithreading
Exclusion rectangles
Taking the dynamic screen values
Integrated interrupters
Test dependencies
Session restore facilities
Test sets inside test plan
Report generation and comparison

The main test directory will be study_guide with study_ndx.xml index file.

#6 Updated by Eugenie Lyzenko over 11 years ago

The simple classical hello_world test has been created(as the place to start learning). This is the fully functional test with using screen reading part to get the prompt line(because it is different on different machines) and embedded Java code to truncate the white spaces. The shared variable "shellPromptText" is being initialized in a separate precondition test file.

#7 Updated by Eugenie Lyzenko over 11 years ago

The question:

What results do I have to put into Redmine here? Should I put the whole project zip file here on every new step? Or every single file? Or upload the current project separately somewhere on GCD shared directory?

#8 Updated by Greg Shah over 11 years ago

You don't need to upload the results to redmine. We want these samples in Bazaar.

On filesrv01, I have created a new /opt/code/harness_repo/ directory. Inside there you will find 2 projects:

/opt/code/harness_repo/harness/ (this one is the main harness code that I have imported into Bazaar - don't use CVS for the harness anymore)

/opt/code/harness_repo/samples/ (this one is for all sample code/baselines)

Please "check out" the samples project. It will be empty. You should add/maintain all of your files to/in that project using the normal means (bzr add, bzr commit...). I would like you to put all of your content inside a sub-directory called study_guide. That way we can have more samples (later) that are separate, but which are still in the samples project.

#9 Updated by Eugenie Lyzenko over 11 years ago

The project and the first sample "Hello World" has been integrated and uploaded into bzr. It might be checked out and run.

#10 Updated by Greg Shah over 11 years ago

I have reviewed the added files. It looks very good.

Please add a text file describing how to setup and run the test plans.

#11 Updated by Eugenie Lyzenko over 11 years ago

The text document study_guide_manual.txt has been added and uploaded into bzr docs subdirectory of study_guide project. The instructions to run and description of the first test have been written to the file.

My plan is to describe all tests in full details and update this file as soon as new tests will be available. Please let me know if this is too verbose to describe every step we do in the test plans.

#12 Updated by Greg Shah over 11 years ago

The new study_guide_manual.txt is perfect! Good job.

#13 Updated by Eugenie Lyzenko over 11 years ago

The study_ndx.xml project index file has been updated in bzr. Adding the detailed comments of every important line/section.

#14 Updated by Eugenie Lyzenko over 11 years ago

The second test has been uploaded in bzr - parallel_test.xml. It demonstrates the simple concurrency case and usage of the separate preconditioned test set.

The manual document has also been updated and uploaded in bzr to include description of what the second test does and how it works.

#15 Updated by Eugenie Lyzenko over 11 years ago

The new test semaphores_test.xml has been uploaded in bzr. It is more complex concurrency test demonstrates the usage of the event and mutex semaphores in multithreading environment with creating the resource and changing it in mutually exclusive mode. All tests are running in parallel.

The manual document has also been changed to include description of the new test.

#16 Updated by Eugenie Lyzenko over 11 years ago

The new feature to repeat the same test several times has been added to semaphores_test.xml. And the test plan file in now in bzr, it was forgotten to be added/committed with all other files of this test.

The samples manual change has been updated in bzr to describe the usage of the repeat ability in test set.

#17 Updated by Eugenie Lyzenko over 11 years ago

The new test plan group_test.xml has been added into bzr. This test demonstrates the usage of the group test option to
1. Guarantee the several tests are being executing sequentially one after another in predefined sequence
and
2. Execute several tests in parallel mode when two or more tests run concurrently on different execution threads.

The manual document update has also uploaded in bzr. This update includes the description of the new group test plan.

#18 Updated by Eugenie Lyzenko over 11 years ago

The new test plan file_report_test.xml has been added to study guide. This is the report file comparison demo. Illustrates the stages of the report generation and comparison. The report reference file, test file and respective screens have been uploaded in bzr.

Also the manual updated to reflect the adding new test plan.

#19 Updated by Eugenie Lyzenko over 11 years ago

The new test plan interrupt_test.xml has been added to the study guide bzr. This test demonstrates the ability to interrupt the child process running in one parallel test by the thread running in another concurrent test.

The study guide manual document has also been updated and uploaded into bzr to reflect the new change.

#20 Updated by Greg Shah over 11 years ago

All planned samples are done at this point, right? If so, I will close this task.

#21 Updated by Eugenie Lyzenko over 11 years ago

Yes, all planned features were provided with samples. But the tests are pretty simple. So if this is OK you can close the task.

#22 Updated by Greg Shah over 11 years ago

  • Status changed from WIP to Closed

Also available in: Atom PDF