GNUrc

From ProgClub
Revision as of 10:55, 11 April 2015 by John (talk | contribs) (→‎v2_exception_log table: previous_id...)
Jump to: navigation, search

GNUrc is the GNU Remote Control software. That's the software that aids in the maintenance of your thermostats. For other projects see projects.

Status

v1.1 released, v2.0 under development.

Motivation

To seek operational efficiencies in air-conditioners thereby saving electricity and improving comfort.

Administration

Contributors

Members who have contributed to this project. Newest on top.

All contributors have agreed to the terms of the Contributor License Agreement. This excludes any upstream contributors who tend to have different administrative frameworks.

Upstream (GNU) contributors: GNU Remote Control Contributors

Copyright

Copyright 2015, Contributors.

License

Licensed under the AGPL.

Resources

Downloads

See the home page for downloads.

Source code

See the home page for source code.

Links

Jargon

Following are definitions for abbreviations and acronyms that we use:

svn
Subversion version control software
i18n
intl
Internationalization/translation
SQL
Structured Query Language
API
Application Programming Interface
MVC
Model/View/Controller
ORM
Object/Relational Mapping
BOM
Business Object Model
HTML
HyperText Markup Language
HTTP
Hypertext Transfer Protocol
XSRF
Cross-Site Request Forgery
AJAX
Asynchronous JavaScript and XML

Database specification

Database data types

You can read more about MySQL data types.

Name MySQL Name Size (Bytes) Min Max
ID8 TINYINT 1 -128 127
ID16 SMALLINT 2 -32,768 32,767
ID24 MEDIUMINT 3 -8,388,608 8,388,607
ID32 INT 4 -2,147,483,648 2,147,483,647
ID64 BIGINT 8 -9,223,372,036,854,775,808 9,223,372,036,854,775,807
Int8 TINYINT 1 -128 127
UInt8 TINYINT UNSIGNED 1 0 255
Int16 SMALLINT 2 -32,768 32,767
UInt16 SMALLINT UNSIGNED 2 0 65,535
Int24 MEDIUMINT 3 -8,388,608 8,388,607
UInt24 MEDIUMINT UNSIGNED 3 0 16,777,215
Int32 INT 4 -2,147,483,648 2,147,483,647
UInt32 INT UNSIGNED 4 0 4,294,967,295
Int64 BIGINT 8 -9,223,372,036,854,775,808 9,223,372,036,854,775,807
UInt64 BIGINT UNSIGNED 8 0 18,446,744,073,709,551,615
Single FLOAT 4
Double DOUBLE 8
Date DATE 3 1000-01-01 9999-12-31
Time TIME 3 -838:59:59 838:59:59
DateTime DATETIME 8 1000-01-01 00:00:00 9999-12-31 23:59:59
Timestamp TIMESTAMP 4 1970-01-01 00:00:01 2038-01-19 03:14:07
Varchar VARCHAR
Longtext LONGTEXT

Database collations

We use the following collations for string data. You can read more about MySQL Character Sets and Collations.

Name MySQL Name Max Bytes per Char Case-sensitive
BINARY binary 1 Yes
ASCII ascii_bin 1 Yes
ASCII_CI ascii_general_ci 1 No
UTF8 utf8_bin 3 Yes
UTF8_CI utf8_general_ci 3 No

Data format specification

Database tables

The database has been completely replaced for v2.0.

v2_exception_log table

Name Type Size Collation Null Default Extra Comment
id ID24 No AUTO_INCREMENT
timestamp Timestamp No CURRENT_TIMESTAMP
previous_id ID24 No 0 previous_id
type Varchar 64 ASCII_CI No type
message Longtext UTF8_CI No message
code Int32 No 0 code
const Varchar 128 ASCII_CI No const
file Varchar 256 ASCII No file
line UInt16 No 0 line
message_stack Longtext UTF8_CI No message_stack
trace Longtext UTF8_CI No trace
trace_data Longtext UTF8_CI No trace_data

v2_exception_log.previous_id column

The ID of the previous exception, if any; otherwise zero.

v2_exception_log.type column

The name of the PHP exception class.

v2_exception_log.message column

The message of the PHP exception.

v2_exception_log.code column

The exception code. When -ve values are error numbers.

v2_exception_log.const column

When code is -ve this field is the ERROR CONST label at the time of the error. See declaring error numbers for details of format.

v2_exception_log.file column

The full path to the PHP file that raised the exception.

v2_exception_log.line column

The line number of the expression that raised the exception.

v2_exception_log.message_stack column

All exception messages in a row. I.e. previous exception messages followed by consequent exception messages.

v2_exception_log.trace column

The callstack at time of exception as a string with one function call per line.

v2_exception_log.trace_data column

The callstack at time of exception as a JSON data structure representing the callstack.

Functional specification

The functional specification describes what the project does.

Administration features

Administration dashboard

The administration dashboard (admin-home.php) is an administrator's home page. From the administration dashboard an administrator can access:

  1. User administration
  2. Translation administration
  3. Group administration
  4. Error reporting

User administration

Translation administration

Group administration

Error reporting

Translation features

Standard features

Role-based security

We use role-based security to limit system functions to particular classes of users. There are three user roles and any given user can be in any combination of the roles:

  1. Administrator
  2. Translator
  3. Standard

Administrators have access to administration functions. Non-administrators do not have access to administration functions. Translators have access to translation functions. Non-translators do not have access to translation functions. All users have access to the standard functions.

Internationalization features

The i18n functional spec is documented on the Language Administration Help page.

Backend features

Error logging

Error log testing

Technical specification

The technical specification describes how the project works.

Model-view-controller

GNUrc 2.0 implements the Model-View-Controller (MVC) design pattern.

Software layers

The GNUrc software is layered:

Layer Directory MVC role
Function Libraries /src/1-lib
Objects /src/2-obj Model/View
Data Access Layer (DAL) /src/3-dal Model
Object/Relational Mapping (ORM) /src/4-orm Model
Business Object Model (BOM) /src/5-bom Model
View helpers /src/6-view View
Ajax Controllers /src/7-ajax Controller
Page Controllers /web Controller

Each level depends on (up to) all of the previous levels. So controllers use views, BOM, ORM, DAL, objects, and modules. Views use BOM, ORM, DAL, objects, and modules. The BOM uses the ORM, DAL, objects, and modules. And so on. Generally controllers should call on the services of the BOM rather than calling on the services of the DAL directly, so the BOM encapsulates business logic and mediates it into the DAL. If higher layers can encapsulate functionality in lower layers that is a good thing to do.

Services and modules

In order to facilitate a team of developers working on the code base at the same time we implement a software module pattern. Note that this is a "political" rather than "technical" or "logical" categorisation of functionality. We separate our code into modules that developers can "own" while they're working on the project. The idea is to avoid changing files that other developers are working on so as to reduce conflicts.

The idea of software modules is to split functionality into parts which can be managed by one developer at a time. For example consider the Data Access Layer (DAL) service. We could have one DAL object that had methods for manipulating thermostats, users, and groups. The problem is that if Miguel is working on thermostats and Federico is working on users, if they both edit the single DAL class then they run the risk of overwriting each others' work. By splitting functionality for thermostats, users, and groups into DAL modules each of our developers can work on their modules without interfering with code used by other developers.

So "services" are an interface into a collection of modules, and "modules" are a bunch of features/functionality implemented by one developer at a time.

The software module pattern is employed by the following services:

Service Modules
DAL /src/3-dal/*/module
BOM /src/5-bom/module
AJAX Controller /src/7-ajax/module
Page Controller /web

Directory structure

The software has the following directory structure starting in the base:

/

The base directory is /path/to/your/gnurc.

  • .svn-ignore -- files for Subversion (svn) to ignore
    e.g.: $ svn propset svn:ignore -RF .svn-ignore .
  • config.example.php -- an example config file
  • config.php -- the production config file
    e.g.: $ cp config.example.php config.php; vim config.php

/dat

The /dat directory is for data files.

/etc

The /etc directory is for miscellaneous scripts.

/etc/dbscripts

The /etc/dbscripts directory is an svn:externals for the database scripts:

svn://svn.sv.gnu.org/remotecontrol/branches/development/dbscripts

/src

The /src directory is for most of the source code.

  • include.php -- the main include file for loading the GNUrc software components.
  • test.php -- the main include file for use by PHPUnit unit tests.

/src/1-lib

The /src/1-lib directory contains function libraries.

/src/2-obj

The /src/2-obj directory contains PHP classes used by GNUrc that do not fit in another category.

/src/3-dal

The /src/3-dal directory contains data access layer facilities.

  • GrcDal.php -- the include file for the Data Access Layer; defaults to MySQL/PDO API.
  • GrcDalModule.php -- a base class for DAL Modules.
/src/3-dal/mysql-pdo

The /src/3-dal/mysql-pdo directory contains data access layer facilities for the MySQL/PDO database connectivity.

/src/3-dal/mysql-pdo/module

The /src/3-dal/mysql-pdo/module directory contains DAL modules.

/src/4-orm

The /src/4-orm directory contains Object/Relational Mappers. The ORM strategy that we employ is known as the Active Record pattern. An Active Record is a class that mediates data between the program (a PHP object) and the database (a record in a table). As a general rule (which can be broken) the Active Record is supposed to be CRUD only, any other business logic and/or validation should be implemented as a business process in the Business Object Model.

/src/5-bom

The /src/5-bom directory contains an interface into "business process" via the Business Object Model (GrcBom):

  • GrcBom -- the business process interface
/src/5-bom/module

The /src/5-bom/module directory contains business process modules accessed via the BOM.

/src/6-view

The /src/6-view directory contains libraries of helper functions for view composition:

/src/7-ajax

The /src/7-ajax directory contains AJAX controllers.

  • GrcAjax -- a service interface into AJAX operations (modules)
/src/7-ajax/module

The /src/7-ajax/module directory contains AJAX modules exposed by the GrcAjax service.

/test

The /test directory contains Unit Tests for the project.

/test/php

The /test/php directory contains test cases that demonstrate a PHP feature.

/test/src

The /test/src directory contains test cases that test a program feature.

/web

The /web directory contains front-facing components. Those are components which are accessible via HTTP(S). Each PHP file in the /web directory is known as a Page Controller. In addition to Page Controllers the /web directory includes:

Managing errors

The error management subsystem is comprised of the /src/2-obj/GrcError.php class and the 'err' and 'Error' functions in /src/1-lib/01-api.php.

Errors are managed in two modes: first they are 'declared' and second they are 'raised'.

Declaring error numbers

To declare a possible error call the err()->define function. The first argument is the name of the error known as the error const. The error const has up to 6 parts separated by double-underscore:

  1. 'ERROR'
  2. type (e.g. 'LIB', 'OBJ', etc.)
  3. module/class (e.g. 'API', 'VALIDATION')
  4. function name
  5. variable name
  6. state (e.g. 'NOT_NULL', 'INVALID')

The second argument is the error message. The error messages will be translated if necessary and can include variables using the '%variable%' notation of the i18n subsystem.

Raising particular errors

After you have declared an error like this:

err()->define( 'ERROR__LIB__EXAMPLE', 'An error with %param% occurred.' );

you can raise it like this:

throw Error( ERROR__LIB__EXAMPLE, 'param', 'value', intl_context, previous_exception );

where intl_context is an i18n translation context and previous_exception is a previous exception if any.

Security features

In addition to role-based security we protect from XSRF and SQL-injection attacks.

XSRF protection

To prevent XSRF attacks we configure a session token that must be included in all HTTP POST submissions.

SQL-injection protection

To prevent SQL injection attacks we take care to escape inputs when building SQL strings and/or use parameters with our database API.

Character encoding

All textual content is encoded, stored, and transmitted as UTF-8.

HTML charset

Read about character encodings in HTML for information about how to declare the UTF-8 charset in various HTML versions.

HTTP Content-Type

When sending HTTP replies the Content-Type should be specified with a charset parameter as described here.

For HTML the Content-Type would be:

Content-Type: text/html;charset=utf-8

For JSON the Content-Type would be:

Content-Type: application/json;charset=utf-8

And so on.

Internationalization (i18n) API

Internationalization functions defined in /src/1-lib/45-intl.php are the interface into the i18n API. They expose functions for the 01n paradigm and for table-heading/HTML/HTML-attribute/text/safe-text content types. See the functional specification for details on the 01n paradigm and content types.

In addition to the above API the HTML composition system supports automatic translation of content via the following methods:

  • GrcHtmlElement->add_nbsp: add HTML after converting spaces to  
  • GrcHtmlElement->add_html: add HTML content (no escaping)
  • GrcHtmlElement->add_h01n: add HTML 01n paradigm
  • GrcHtmlElement->add_attr: add HTML attribute content (no escaping, angle brackets should not be present)
  • GrcHtmlElement->add_a01n: add HTML attribute 01n paradigm
  • GrcHtmlElement->add_text: add text content (HTML characters escaped)
  • GrcHtmlElement->add_t01n: add text 01n paradigm
  • GrcHtmlElement->add_safe: add safe text content (content must not include HTML special chars)
  • GrcHtmlElement->add_s01n: add safe text 01n paradigm

Web interface

admin-home.php

The admin-home.php

Notes

Notes for implementers

If you are interested in incorporating this software into your project, here's what you need to know:

Notes for developers

If you're looking to set up a development environment for this project here's what you need to know:

Tasks

TODO

Things to do, in rough order of priority:

  • Document system design
  • Generate task list
  • Allocate tasks to developers

Done

Stuff that's done. Latest stuff on top.

  • JE 2015-04-01: created project page