User Tools

Site Tools


wiki:software_assessment

Software assessment

1) Meta

  • 1.1) Does the software have a good name?
    • All software projects should start with a good name. Bonus points for animals or (recursive) acronyms! ;)
  • 1.2) What is the license? Is it open source? Is the license clearly stated? Is it a good choice?
  • 1.3) Are author(s) and affiliation(s) clearly stated?
  • 1.4) Is the software maintained in proper versioning control? (If not, flog the programmer)
    • Is the versioning control public? Is the version control system a good choice?
  • 1.5) Is the purpose of the software clear?
    • Does the software have a website? Does the site properly explain what it does?
  • 1.7) Is the software easily downloadable?
  • 1.8) Is the intended audience clear?

2) Installation

  • 2.1) Does the software have a clear and generic installer?
    • Is it an ad-hoc or a platform-standard installer?
    • Does the installer make rigid assumptions about where libraries/data are located?
  • 2.2) Does it depend on many other libraries and how hard is it install these necessary dependencies?
  • 2.3) Does it compile/install?
    • How clean is compiler/installer output, does it generate many warnings?
  • 2.4) How does it fare on other platforms, such as Mac OS X or *shiver* Windows?

3) Status

  • 3.1) How mature is the software? Is it alpha, beta, release candidate, stable?
  • 3.2) What can be expected at this stage?
  • 3.3) Does it do what it claims/promises to do?
  • 3.4) What parts need more development?

4) Documentation

  • 4.1) Is there any documentation? (If not, flog the programmer)
    • Is it clear enough for the intended audience?
    • Is it complete?
  • 4.2) If the sofware offers an API, is there an API reference?
    • Is it clear and verbose enough?
    • Is it complete?
    • Are there enough examples?
    • Is the API documentation generated from the code itself?
  • 4.3) Is there a hand-on tutorial for working with the sofware?
    • ..as an end-user?
    • ..as a programmer?

5) Software Architecture

  • 5.1) Programming Language
    • What programming language(s) is/are used? Are they good choices for the task? Is the language well-supported and expected to last?
    • What libraries and frameworks are used? Are they good choices for the task? Are there libraries/frameworks missing which could have been used?
  • 5.2) Form
    • What is the form of the software: Web-based application, Webservice, Command line tool, Programming Library, GUI? Is this a suitable choice (considering the intended audience)? Should it be expanded?
  • 5.3) Algorithms & Techniques
    • What algorithms and techniques are used?
    • Are these good for the task at hand? Can they be improved?
  • 5.4) User Interface
    • Is there a GUI or web-interface? If not, could/should there be?
    • Is the interface intuitive?
    • Does the interface expose enough of the underlying functionality?
    • Is the interface visually appealing?
    • For web-based interfaces: How is cross-browser support?
  • 5.5) Application Programming Interface
    • Is there an API interface? If not could/should there be? For what languages?
    • Are classes/methods etc suitably named?
  • 5.6) Data
    • Does the software have sufficient facilities for reading and storing data?
    • Does the software define any new formats for its data?
      • Are these sufficiently documented?
      • Does it follow standards (XML), or is it too ad-hoc? Is there a scheme? (XSD, RelaxNG)
      • Is it a good choice?
    • Are there enough import/export facilities for data in other formats? Should there be?
    • How is the visualisation of the data?
    • Are the data metrics used by the software good choices?
  • 5.7) Unit tests
    • Are there unit tests/regression tests?
    • Are they any good?
    • Do they pass?
    • Do they cover enough of the software?
  • 5.8) Security
    • How is the security of the software, are there any obvious security holes?
    • Is there, or should there be, proper user authentication? Are user credentials properly encrypted and transferred over the net as such?
    • Is user input in the interface properly validated?
    • Is the software sufficiently hardened against SQL injection, shell injection and cross-site scripting?
    • Is any sensitive data leaking? Is there anonimisation or data encryption, should there be?
  • 5.6) Modularity [see also 6.1]
  • Is the code split into proper modules? Are these reusable for other ends? Should the software be split into parts?
  • 5.7) Performance [see also 6.3]
    • How is the memory usage of the sofware? Does this seem right?
    • How quickly does the software perform its task? Does this seem right?
  • 5.8) Exception Handling [see also 6.4]
    • Is the error output sane and understandable? Can the user easily remedy the problem?
  • 5.9) Maintainability
    • How maintainable is the software?
    • Is it build to last?
    • Can it survive if the original programmer(s) abandon it?
    • Is the software easily extendible with new functionality?
    • Are there needless complexities or over-simplifications? Should parts be simplified, or generalised?

6) Source review

  • 6.1) Modularity [see also 5.6]
    • Is the code object oriented, should it be? Are the choices logical?
    • Think about: seperation into classes, proper constructors,
    • method/operator overloading, class hierarchy, polymorphism / multiple inheritance
    • Is the code split into clear functions? Is code repetition kept to a strict minimum?
    • Are global variables kept to a strict minimum?
    • Are settings probably configurable and not hard-coded (paths,addresses
  • 6.2) Style & Comments
    • Is the coding style and naming clear and clean?
    • Does the coding style and naming follow standard conventions for the language?
    • Are there comments to explain what each function/class does?
    • Are there enough comments to explain difficult parts?
    • Are standard or third-party libraries used properly and enough, or is the wheel reinvented needlessly?
    • Are there compatibility issues for different versions of the language?
  • 6.3) Performance [see also 5.7]
    • Is allocated memory properly freed when no longer needed? Think about: Dangling references/pointers
    • Are the algorithms efficient enough for the task? Are there better options? Think about: Nested loops
    • Is the speed/memory trade-off decent?
  • 6.4) Exception Handling [see also 5.8]
    • Is there enough exception handling? Are obvious chokepoints ignored?
    • Is user input properly validated?
    • Is the software verbose enough in raising errors of its own?
    • Is there any obvious danger of segmentation faults, null pointer exceptions, double free or overflowing data types?
  • 6.5) Bugs
    • Are there known bugs?
    • Did you encounter any bugs? (If not, you didn't look hard enough)
    • What should be prioritised when fixing?

7) Case Study

  • 7.1) Do you see any use for the software in your own project(s)?
  • 7.2) Any interesting ideas for projects that could be conducted with this software?
wiki/software_assessment.txt · Last modified: 2014/09/04 14:40 by Wessel Stoop