wiki:Software/EventBenchConsole

Version 20 (modified by sherbold, 13 years ago) (diff)

--

EventBenchConsole

TracNav

Table of Contents

  1. Recent Changes
  2. Requirements
  3. Description
  4. Exemplary Work-flows
  5. Commands
    1. Commands for handling data generated by [wiki:Software/userlog …
    2. Commands for handling data generated by [wiki:Software/PHPMonitor …
    3. Commands for working with ​GUITAR
  6. Translating MFC Messages into Events
    1. Single Messages and Sequences
    2. Storage of messages
    3. Checking equalities
    4. Generation of replay sequences

Recent Changes

Error: Macro ChangeLog(/trunk/EventBenchConsole, 3) failed
'NoneType' object has no attribute 'get_youngest_rev'

Requirements

Description

A console application that uses the EventBenchCore. It provides three translation layers:

  1. Windows MFC applications logged with the MFCUsageMonitor and for replaying with MFCReplay
  2. PHP based web applications logged with PHPMonitor and replaying with ​curl
  3. To the ​GUITAR project.

Exemplary Work-flows

Creating a based on monitored sessions: 

convertToXml logfile.txt log.xml
parseXML log.xml
generateReplayfile replay.xml

Creating a randomly generated replay session based on a first-order Markov model: 

convertToXml logfile.txt log.xml
parseXML log.xml
trainMarkovModel markovModel
generateRandomReplay markovModel replay.xml

Commands

  • calcCoverage <modelname> <observedSequences> [sequenceNames] <minCovLength> <maxCovLength>
    • Calculates the coverage criteria.
    • Examples:
      • calcCoverage markovModel trainingSequences [generatedSequences_1 generatedSequences_2] 2 3
  • calcEntropy <modelname>
    • Calculates the entropy of a first-order Markov model.
    • Examples:
      • calcEntropy markovModel
  • deleteObject <objectname>
    • Deletes an object from the storage.
    • Examples:
      • deleteObject markovModel
      • deleteObject sequences
  • generateFixedLengthSequences <modelname> <sequencesName> <minlenght> <maxlength> {<all>} {<numSequences>}
    • Generates sequences of a predefined length. This command first generates all possible sequences of the defined lengths. In case <all> is true, all generated sequences are then stored. In case <all> is false <numSequences> sequences will be randomly drawn from all the sequences, based on their probability according to <modelname>. The default value for <all> is true.
    • Examples:
      • generateFixedLengthSequences markovModel generatedSequences 3 5
      • generateFixedLengthSequences markovModel generatedSequences 3 5 true
      • generateFixedLengthSequences markovModel generatedSequences 3 5 false 1000
  • generateRandomReplay <modelname> <filename> {<numSessions>}
    • Generates a randomly generated replay from a previously learned usage model. The default number of sessions generate is 1. With this command, is it not possible to configure the length of the generated sequences.
    • Examples:
      • generateRandomReplay markovModel replay.xml
      • generateRandomReplay markovModel replay.xml 5
  • generateRandomSequenecs <modelName> <sequencesName> <numSessions> <minLength> <maxLength> {<maxIter>}
    • Generates sequences of a predefined length. This command randomly generates sequences from the usage profile. In case the length of a generated session has the desired length, it is added to the set of sequences. The maximum number of randomly generated sequences is defined by <maxIter>, which is by default <numSessions>*10.
  • generateReplayfile <filename> <sequences>
    • Generates a replay file from a set of sequences.
    • Examples:
      • generateReplayfile d:/data/replay.xml sequences
  • listStates <modelName> {<sort>}
    • Lists all states of a stochastic process. If <sort> is true, the states will be sorted alphabetically.
    • Examples:
      • listStates markovModel
      • listStates markovModel true
  • load <filename>
    • Loads a data container from a file. All currently stored data is discarded.
    • Examples:
      • load d:/data.dat
  • loadObject <filename> <objectName>
    • Loads an object from a file to the storage.
    • Examples:
      • loadObject d:/markovModel.obj markovModel
  • modelSize <modelname>
    • Prints information about the size of a usage profile.
    • Examples:
      • modelSize markovModel>
  • printDot <modelname>
    • Prints the ​Dot graph representation of a model to the console.
    • Examples:
      • printDot markovModel
  • printTrieDot <modelname>
    • Prints the ​Dot graph representation of a trie used by a model to the console.
    • Examples:
      • printTrieDot markovModel
      • printTrieDot predictionByPartialMatch
  • save <filename>
    • Saves all currently stored data to a file.
    • Examples:
      • save d:/data.dat
  • saveObject <filename> <objectname>
    • Saves a single object from the storage to a file.
    • Examples:
      • saveObject d:/markovModel.obj markovModel
  • sequenceStatistics <sequencesName>
    • Prints statistics about a collection of sequences.
    • Examples:
      • sequenceStatistics sequences
  • showMarkovModel <modelname> {<showNodeNames>}
    • Opens a window that displays a first-order Markov model as a directed graph. Per default, the node names are not shown, as the graph gets very ugly and has overlapping nodes if they are shown.
    • Examples:
      • showMarkovModel markovModel
      • showMarkovModel markovModel true
  • showSequences <sequencesName>
    • Opens a dialog that display a list of all sequences that are part of the current sequence set, including their sizes. The sequences can be edited using this dialog.
  • showTimer <timerName>
    • Prints how many milliseconds elapsed since the start of the time.
    • Examples:
      • showTimer timer
  • showTrie <modelname>
    • Opens a window that displays a trie underlying a stochastic process.
    • Examples:
      • showTrie markovModel
      • showTrie predictionByPartialMatch
  • startTimer <timerName>
    • Starts a timer.
    • Examples:
      • startTimer timer
  • trainDFA <modelname> <sequencesName>
    • Trains a Deterministic Finite Automaton (DFA) from a collection of sequences.
    • Examples:
      • trainDFA dfa trainingSequences
  • trainMarkovModel <modelName> <sequencesName> {<order>}
    • Trains a Markov model based from a collection of seqeuences. The default <order> of the model is 1.
    • Examples:
      • trainMarkovModel markovModel trainingSequences
      • trainMarkovModel markovModel trainingSequences 3
  • trainPPM <modelName> <sequencesName> <probEscape> <maxOrder> {<minOrder>}
    • Trains a Prediction by Partial Match (PPM) model based from a collection of sequences. The default <minOrder> is 0, i.e., random selection.
    • Examples:
      • trainPPM PPMModel trainingSequences 0.01 3
      • trainPPM PPMModel trainingSequences 0.01 3 1
  • updateModel <modelname> <sequencesName>
    • Updates a usage profile with a collection of sequences. This reinforces the model, i.e., it is not completly retrained, but the probabilities are merely updated using the new information.
    • Examples:
      • updateModel markovModel newTrainingSequences

Commands for handling data generated by MFCUsageMonitor

  • convertToXml <filenameSource> <filenameTarget> {<base64>}
    • Converts a logfile with prefixes created by the MFCUsageMonitor into an XML file containing the logged sessions.
    • Examples:
      • convertToXml d:/logfile.txt d:/log.xml
      • convertToXml d:/logfile.txt d:/log.xml true
  • convertDirToXml <directoryNames> <filenameTarget> {<base64>}
    • Converts all files in a directory into one XML file containing all sessions, the same way convertToXml works.
    • Examples:
      • convertDirToXml d:/logdir d:/log.xml
      • convertDirToXml d:/logdir d:/log.xml true
  • parseXML <filename> {<countMessageOccurences>}
    • Parses an XML file containing users sessions. If countMessageOccurences is true, a statistic containing how often each message occurred in the log is printed.
    • Examples:
      • parseXML d:/log.xml
      • parseXML d:/log.xml true

Commands for handling data generated by PHPMonitor

  • loadWebSequences <filename> <sequencesName> {<serverUrl>} {<timeout> <minSessionLength> <maxSessionLength>} {<frequentUserThreshold>}
    • Parses a logfile generated by PHPMonitor and creates a collection of sequences from it. The default <timeout> is 3600000 milliseconds, the default <minSessionLength> is 2, the default <maxSessionLength> is 100. In case a <frequenceUserThreshold> is defined, additional collection of sequences will be defined for all users that have more sessions than this threshold, that fulfill the session length requirements.
    • Examples:
      • loadWebSequences d:/websessions.log web-sequences
      • loadWebSequences d:/websessions.log web-seqeunces http://www.swe.informatik.uni-goettingen.de
      • loadWebSequences d:/websessions.log web-seqeunces http://www.swe.informatik.uni-goettingen.de 7200000 4 50
      • loadWebSequences d:/websessions.log web-seqeunces http://www.swe.informatik.uni-goettingen.de 7200000 4 50 20

Commands for working with ​GUITAR

  • efgTestCasesToSequences <folder> <sequencesName> {<efgFileName>}
    • Generates a collection of sequences from GUITAR test cases. If an Event-Flow Graph (EFG) file is defined, it will be used to resolve the widget IDs associated with the events in the test cases. The widget IDs are used as target for the events.
    • Examples:
      • efgTestCasesToSequences d:/efg-testcases testSequences
      • efgTestCasesToSequences d:/efg-testcases testSequences d:/application.efg
  • efgToDFA <filename> <modelname>
    • Creates a Deterministic Finite Automaton (DFA) from an Event-Flow Graph (EFG).
    • Examples:
      • efgToDFA d:/application.efg dfa
  • efgToMM <filename> <modelname>}
    • Creates a first-order Markov model from an Event-Flow Graph (EFG). The model is such that all possible following events are equally likely. To change that, the updateModel command with a collection of sequences obtained using <efgTestCaseToSequences command can be used.
    • Examples:
      • efgToMM d:/application.efg markovModel

Translating MFC Messages into Events

The messages are aggregated into events using a set of rules defined in XML. The order of the rules in the XML file defines their priority.

Single Messages and Sequences

There is a distinction between single messages and whole sequences of messages of the same type. The former may be used for, e.g., WM_LBUTTONDOWN and WM_LBUTTONUP: 

<msg type="&WM_LBUTTONDOWN;"/>
<msg type="&WM_LBUTTONUP;"/>

Sequences of the same message occur, e.g., when scrolling: 

<msg type="&WM_LBUTTONDOWN;"/>
<msg type="&WM_HSCROLL;" multiple=true/>
<msg type="&WM_LBUTTONUP;"/>

The distinction is using the multiple attribute of the msg node, which is per default false. If it is true, it means at least one, similar to + in regular expressions.

Storage of messages

Each message or message sequence can be stored for later use, e.g., comparisons with other messages or generating replays. This is done using the store and storeSeq nodes: 

<msg type="&WM_LBUTTONDOWN;">
  <store var="clicked"/>
</msg>
<msg type=&WM_HSCROLL;" multiple=true>
  <storeSeq varSeq="scrolls/>
</msg>

When storing a message, the window.hwnd is automatically resolved and its target string is stored as an attribute of the message (winParamType). If other parameters contain HWNDs that need to be later on to address a target, they can be resolved manually and stored as a further parameter: 

<msg type=&WM_HSCROLL;" multiple=true>
  <storeSeq varSeq="scrolls>
    <resolveHwnd param="scrollBarHandle" storeParam="scrollBarTarget"/>
  </storeSeq>
</msg>

Checking equalities

Only matching messages by their type is insufficient. Comparisons between their parameters are also required. This can be done using equals nodes: 

<msg type="&WM_LBUTTONDOWN;">
  <store var="clicked"/>
</msg>
<msg type="&WM_LBUTTONUP;">
  <equals>
    <paramValue obj="this" param="window.hwnd"/>
    <paramvalue obj="clicked" param="window.hwnd/>
  </equals>
</msg>

In the example above, the equals node compares whether the value of the parameter window.hwnd is equal for the stored variable clicked and the current message itself, i.e., this.

In general, each equals node has the same structure: 

<equals>
  termNode1
  termNode2
</equals>

There are four types of term nodes:

  1. paramValue: of the messages (example above)
  2. constValue: constant value, e.g., <constValue value="Button"/>
  3. winInfoValue: information about the target of the message.
    • <winInfoValue obj="this" winParam="class"/> for the class of the target
    • <winInfoValue obj="this" winParam="hwnd"/> for the HWND
    • <winInfoValue obj="this" winParam="resourceId"/> for the resource Id.
    • <winInfoValue obj="this" winParam="parentTarget"/> for the target of the GUI objects parent
  4. msgInfoValue: information about the type and target of the message.
    • <msgInfoValue obj="this" msgParam="type"/> for the type of the message (its integer)
    • <msgInfoValue obj="this" msgParam="target"/> for the target of the message (its string representation)

The comparison of sequence variables is not yet implemented!

Generation of replay sequences

If a message sequence has been successfully matched to a rule, a replay sequence is generated. This is done using genMsg and genMsgSeq nodes.

The first way to generate a replay message is to directly replay a message that has been stored during the matching phase: 

<genMsg delay="100">
  <storedVar obj=clicked"/>
</genMsg>

The delay attribute signifies that the replaying tool pauses for 100 ms after sending the message stored in the variable clicked.

The second way is to construct it: 

<genMsg delay="100">
  <type>
    <msgInfoValue obj="cmd" msgParam="type"/>
  </type>
  <target>
    <msgInfoValue obj="cmd" msgParam="target"/>
  </target>
  <LPARAM>
    <paramValue obj="cmd" param="sourceDesc"/>
  </LPARAM>
  <WPARAM>
    <paramValue obj="cmd" param="WPARAM"/>
  </WPARAM>
</genMsg>

The type needs to be an integer. The target needs to be a target string. The LPARAM and WPARAM are both optional, their default values are 0. They can be either numerical or target strings. Furthermore, it is possible to define the HI and LO word of them separatly: 

<LPARAM>
  <LOWORD>
    value
  </LOWORD>
  <HIWORD>
    value
  </HIWORD>
</LPARAM>

The value types are the same that can be used as terms for equals nodes.

To generate message sequences, the genMsgSeq node is used: 

<genMsgSeq delay="20">
  <type>
    <constValue value="&TBM_SETPOS;"/>
  </type>
  <target>
    <seqValue seqObj="scrolls" param="scrollBarTarget"/>
  </target>
  <LPARAM>
    <constValue value="1"/>
  </LPARAM>
  <WPARAM>
    <seqValue seqObj="scrolls" param="scrollPos"/>
  </WPARAM>
</genMsgSeq>

The three value types used for equals are allowed. However, either the type or the target must be of a seqValue, otherwise the creation will fail. In case more than one seqValue variable is used, e.g., a different one for the LPARAM than for the target, both sequences must have the same length.