Evaluation methodology for comparing memory and communication of analytic processes in visual analytics

Eric D. Ragan, John R. Goodall

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

8 Scopus citations

Abstract

Provenance tools can help capture and represent the history of analytic processes. In addition to supporting analytic performance, provenance tools can be used to support memory of the process and communication of the steps to others. Objective evaluation methods are needed to evaluate how well provenance tools support analysts' memory and communication of analytic processes. In this paper, we present several methods for the evaluation of process memory, and we discuss the advantages and limitations of each. We discuss methods for determining a baseline process for comparison, and we describe various methods that can be used to elicit memory of an analysis for evaluation. Additionally, we discuss methods for conducting quantitative and qualitative analyses of process memory. We discuss the methodology in the context of a case study in using the evaluation methods for a user study. By organizing possible memory evaluation methods and providing a meta-analysis of the potential benefits and drawbacks of different approaches, this paper can inform study design and encourage objective evaluation of process memory and communication.

Original languageEnglish
Title of host publicationProceedings - Beyond Time and Errors
Subtitle of host publicationNovel Evaluation Methods for Visualization, BELIV 2014 - 5th Bi-Annual Workshop
EditorsPetra Isenberg, Tobias Isenberg, Michael Sedlmair, Heidi Lam
PublisherAssociation for Computing Machinery
Pages27-34
Number of pages8
ISBN (Electronic)9781450332095
DOIs
StatePublished - Nov 10 2014
Event5th Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, BELIV 2014 - Paris, France
Duration: Nov 10 2014 → …

Publication series

NameACM International Conference Proceeding Series
Volume10-November-2015

Conference

Conference5th Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, BELIV 2014
Country/TerritoryFrance
CityParis
Period11/10/14 → …

Funding

This manuscript has been authored by UT-Battelle, LLC, under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.

Keywords

  • Analytic provenance
  • Evaluation
  • Process memory
  • Visual history

Fingerprint

Dive into the research topics of 'Evaluation methodology for comparing memory and communication of analytic processes in visual analytics'. Together they form a unique fingerprint.

Cite this