Commit 4722d70d authored by 's avatar
Browse files

Mostly update of README, some small code updates


git-svn-id: file:///home/svn/mapi/trunk@342 8d5bb341-7cf1-0310-8cf6-ba355fef3186
parent 01e9a07d
......@@ -9,9 +9,9 @@ LIB_DIRS=
RLIB_DIRS=
INCLUDE_DIRS=
TARGETS=abw abw_dummy test_abw_time test_config
OBJS=abw.o abw_mysql.o abw_time.o abw_dummy.o test_abw_time.o test_config.o \
abw_conf.o
TARGETS=abw_measure abw_dummy test_abw_time test_config
OBJS=abw_measure.o abw_mysql.o abw_time.o abw_dummy.o test_abw_time.o \
test_config.o abw_conf.o
all: $(TARGETS)
......@@ -22,7 +22,7 @@ all: $(TARGETS)
# $(AR) rv $@ $(LIBSCAMPI_OBJS)
# $(RANLIB) $@
abw: abw.o abw_mysql.o abw_time.o abw_conf.o
abw_measure: abw_measure.o abw_mysql.o abw_time.o abw_conf.o
$(LD) -o $@ abw.o abw_mysql.o abw_time.o abw_conf.o $(LIB_DIRS) $(LIBS) -Wl,-rpath,$(RLIB_DIRS)
abw_dummy: abw_dummy.o
......
......@@ -68,128 +68,131 @@ Design decisions:
interface, for this purpose, it is most convenient to store this configuration
in SQL database
- we decided to develop the following programs:
- Prototype C program abw:
- Prototype C program abw reads measurement configuration from
configuration file abw.cfg or from command-line, does measurements
and prints results on stdout and stores them in MySQL database
- abw reads measurement configuration from configuration file or from
command-line, does measurements and prints results on stdout and stores
them in MySQL database
command-line --> ----- --> stdout
conf.file --> ----- --> stdout
| abw |
file abw.cfg --> ----- --> results in MySQL
command-line --> ----- --> results in MySQL
abw runs on the monitoring node. With DiMAPI it can also run on
central machine and retrieve measurement data from measurement nodes
using DiMAPI.
- abw runs on the monitoring node. With DiMAPI it can also run on the
central machine and retrieve measurement data from measurement nodes
using DiMAPI.
abw does not allow to run measurements continuosly and read only
selected results and does not allow web-based editing of measurement
configuration.
- abw does not allow to run measurements continuosly and read only
selected results and does not allow web-based editing of measurement
configuration.
Therefore, we separated the whole process into several parts
- Separation of the whole process into several parts:
- processing measurement configuration
- doing measurements and storing raw results
- aggregating results
- retrieving results
- processing measurement configuration
- doing measurements and storing raw results
- aggregating results
- retrieving results
- PHP scripts read requests on web interface and read, edit and store
measurement configuration in MySQL database, a signal is then sent
to the application doing measurements
- Interaction of components:
-------------
web interface <--> | PHP scripts | <--> configuration in MySQL
-------------
|
v
abw_measure
-------------------
web UI <--> | PHP conf. scripts | <----
------------------- |
conf. file --> ---------- v
| abw_conf | --> conf. in MySQL --> ------------- --> stdout
command-line --> ---------- | abw_measure |
command-line --> ------------- --> results
in MySQL
|
-------------------- |
web UI <--> | PHP result scripts | <--
-------------------- |
|
request on command-line or in conf. file --> ----------- |
| abw_print | <---
stdout <-- -----------
PHP scripts run on central machine.
- Description of components:
- C program abw_conf treads configuration file abw.cfg or command-line
arguments and adds or replaces configuration in MySQL database, a signal
is then sent to the application doing measurements
- abw_conf reads measurement configuration from configuration file or
command-line and writes configuration to MySQL database, a signal is
then sent to abw_measure
command-line --> ----------
| abw_conf | --> configuration in MySQL
file abw.cfg --> ----------
|
v
abw_measure
- PHP conf. scripts read measurement configuration on web interface
and read and write configuration to MySQL database, a signal is then
sent to abw_measure
abw_conf runs on central machine.
- abw_measure reads measurement configuration from MySQL database or
command-line, does the measurements and prints results on stdout
or stores them to MySQL database
- C program abw_measure reads configuration from MySQL database or
command-line (for easier testing), does the measurements and prints
results to stdout or stores them in MySQL database
- abw_print reads requests for results from configuration file or
command-line, reads results from MySQL database and prints results
on stdout
command-line --> ------------- --> stdout
| abw_measure |
configuration in MySQL --> ------------- --> results in MySQL
- PHP result scripts read requests for results from web interface, read
results from MySQL database and present them on web interface
abw_measure runs on measurement nodes, reads configuration remotely
from MySQL on central machine and stores results in local MySQL.
With DiMAPI it can also run on central machine and retrieve measurement
data from measurement nodes using DiMAPI.
Where the components run:
- PHP scripts read results from MySQL database and present them on
web interface
If we have one central machine and measurement nodes, components run
as follows:
-------------
web interface <--> | PHP scripts | <-- results in MySQL
-------------
- abw_measure runs on measurement nodes, reads configuration remotely
from MySQL on the central machine and stores results in local MySQL.
PHP scripts run on central machine and retrieve results remotely
from MySQL on meaurement nodes or from local MySQL (such as
aggregated results)
With DiMAPI abw_measure can also run on the central machine and retrieve
measurement data from measurement nodes using DiMAPI.
- C program abw_print reads request for results from configuration file
abw.cfg or from command-line and prints results on stdout
- abw_conf and PHP conf. scripts run on the central machine
----------- <-- request from abw.cfg or command-line
stdout <-- | abw_print |
----------- <-- results in MySQL
- abw_print and PHP result scripts run on the central machine and retrieve
results remotely from MySQL on meaurement nodes or from local MySQL (such
as aggregated results or when DiMAPI is used)
abw_print runs on the same nodes as PHP scripts.
Format of results printed on stdout is as follows:
Format of results printed on stdout is as follows:
<parameters_id> <seconds.microseconds> <packets>/<bytes> ...
<parameters_id> <seconds.microseconds> <packets>/<bytes> ...
Where <parameters_id> is equal to "id" field of [parameters]
section in measurement configuration, <seconds.microseconds>
is timestamp corresponding to the current line and <packet>/<bytes>
is the numner of packets and bytes measured for some characteristics.
There is one <packet>/<bytes> pair for each characteristics
specified in "characteristics" field in measurement configuration.
Where <parameters_id> is equal to "id" field of [parameters]
section in measurement configuration, <seconds.microseconds>
is timestamp corresponding to the current line and <packet>/<bytes>
is the numner of packets and bytes measured for some characteristics.
There is one <packet>/<bytes> pair for each characteristics
specified in "characteristics" field in measurement configuration.
Data aggregation:
- C program abw_aggregate reads aggregation configuration from MySQL
database or from command-line (for easier testing) and aggregates
results in MySQL database
- abw_aggregate reads aggregation configuration from MySQL database or
command-line and aggregates results to MySQL database
command-line --> --------------- <-- raw results in MySQL
| abw_aggregate |
configuration in MySQL --> --------------- --> aggregated results
in MySQL
command-line --> --------------- <-- raw results in MySQL
| abw_aggregate |
configuration in MySQL --> --------------- --> aggregated results
in MySQL
We assume that PHP scripts, abw.cfg and abw_conf will be extended to
also support aggregation configuration.
- We assume that PHP scripts, abw.cfg and abw_conf will be extended to
also support aggregation configuration.
abw_aggreate can run on measurement nodes or on central machine and
works on the same MySQL or reads raw results from one MySQL and stores
aggregated results in another MySQL
- abw_aggreate can run on measurement nodes or on central machine and
works on the same MySQL or reads raw results from one MySQL and stores
aggregated results in another MySQL
- The main program for interface with perfSONAR or with Stager is abw_print.
When using Stager abw_aggregate is not necessary and can be provided by
Stager aggregate.pl script.
Interface with perfSONAR and Stager:
- File abw.cfg is used for two purposes:
- The main program for interface with perfSONAR or with Stager is abw_print.
When using Stager abw_aggregate is not necessary and can be provided by
Stager aggregate.pl script.
- for measurement configuration
- for requests for results
Measurement configuration file (usually abw.cfg):
The syntax is the same for both purposes. For measurement configuration,
start_time_string, end_time_string and interval specify time span and
frequency when measurements are done. For requests for results, these
fields specify time span and frequency of requested results.
It is used for two purposes:
- for measurement configuration
- for requests for results
The syntax is the same for both purposes. For measurement configuration,
start_time_string, end_time_string and interval specify time span and
frequency when measurements are done. For requests for results, these
fields specify time span and frequency of requested results.
[global]
db=abw
user=abw
passwd=lab10
# [node] section specifies where the measurement should be done. You should
# specify two things: 1) hostname or IP address, 2) device file. Multiple
# [node] sections can be specified for several measurements done on diferent
......
......@@ -24,14 +24,17 @@ int read_conf_file(char *filename, flow_spec_t **flow_spec)
return -1;
}
/* Parse configuration file and load its content into a sequence of
structures representing categories and their items in the file */
if (pc_load(filename)!=1) {
fprintf(stderr, "%s: pc_load() did not load file %s\n", __func__,
filename);
return -1;
}
/* Go over all categories and for each [parameters] category create
one flow specification */
cat=conf;
pp=flow_spec;
p=*pp; i=0;
......@@ -44,6 +47,9 @@ int read_conf_file(char *filename, flow_spec_t **flow_spec)
return -1;
}
/* First flow_spec_t structure was already created in main() function
of abw_measure for optionally specified flow by command-line
arguments, following flow_spec_t structures must be created here */
if (p==NULL) {
if ((*pp=new_flow_spec())==NULL) {
fprintf(stderr, "%s: malloc() failed\n", __func__);
......@@ -52,6 +58,9 @@ int read_conf_file(char *filename, flow_spec_t **flow_spec)
p=*pp;
}
/* Retrieve items of the [parameters] section and copy them into the
flow specification. If something is missing or incorrect, it will
detected later by check_conf() function */
node_id=-1;
param=cat->params;
while (param) {
......@@ -144,6 +153,10 @@ int read_conf_file(char *filename, flow_spec_t **flow_spec)
param=param->next;
}
/* Each [parameter] section should refer to one [node] section,
which we have to find, so go over all sections and if it is
a [node] section, check whether it is the one we are lokking for */
if (node_id<0) {
fprintf(stderr, "%s: missing node_id\n", __func__);
return -1;
......@@ -194,14 +207,21 @@ int read_conf_file(char *filename, flow_spec_t **flow_spec)
return 0;
} /* read_conf_file() */
int check_conf(flow_spec_t **flow_spec) {
int check_conf(global_spec_t *global_spec, flow_spec_t **flow_spec) {
flow_spec_t **pp;
flow_spec_t *p;
int i;
if (flow_spec==NULL)
if (global_spec==NULL || flow_spec==NULL)
return 0;
if (global_spec->db) {
if (!global_spec->username || !global_spec->passwd) {
fprintf(stderr, "%s: if MySQL database name is given, you also have to specify MySQL username and password\n", __func__);
return -1;
}
}
pp=flow_spec;
p=*pp; i=0;
while (p) {
......@@ -211,31 +231,36 @@ int check_conf(flow_spec_t **flow_spec) {
return -1;
}
if (p->sau_mode=='p') {
if (p->sau_pass_probability<0 || p->sau_pass_probability>1) {
fprintf(stderr, "%s: flow[%d]: probability must be between 0 and 1\n", __func__, i);
return -1;
}
p->sau_threshold=0xFFFFFFFF * (1 - p->sau_pass_probability);
p->sau_mode_encoded=COMBO6_PROBABILISTIC;
}
else if (p->sau_mode=='b') {
if (p->sau_byte_threshold<0) {
fprintf(stderr, "%s: flow[%d]: byte threshold cannot be negative\n",
__func__, i);
return -1;
}
p->sau_threshold=p->sau_byte_threshold;
p->sau_mode_encoded=COMBO6_LENGTH_DETERMINISTIC;
}
else {
if (p->sau_packet_threshold<0) {
fprintf(stderr, "%s: flow[%d]: packet threshold cannot be negative\n", __func__, i);
return -1;
}
p->sau_threshold=p->sau_packet_threshold;
p->sau_mode_encoded=COMBO6_DETERMINISTIC;
}
/* In configuration file sau_threshold is specified. On command-line,
sampling-type-specific value is specified and sau_threshold is
computed */
if (p->sau_threshold<0) {
if (p->sau_mode=='p') {
if (p->sau_pass_probability<0 || p->sau_pass_probability>1) {
fprintf(stderr, "%s: flow[%d]: probability must be between 0 and 1\n", __func__, i);
return -1;
}
p->sau_threshold=0xFFFFFFFF * (1 - p->sau_pass_probability);
p->sau_mode_encoded=COMBO6_PROBABILISTIC;
}
else if (p->sau_mode=='b') {
if (p->sau_byte_threshold<0) {
fprintf(stderr, "%s: flow[%d]: byte threshold cannot be negative\n",
__func__, i);
return -1;
}
p->sau_threshold=p->sau_byte_threshold;
p->sau_mode_encoded=COMBO6_LENGTH_DETERMINISTIC;
}
else {
if (p->sau_packet_threshold<0) {
fprintf(stderr, "%s: flow[%d]: packet threshold cannot be negative\n", __func__, i);
return -1;
}
p->sau_threshold=p->sau_packet_threshold;
p->sau_mode_encoded=COMBO6_DETERMINISTIC;
}
}
if (strptime(p->start_time_string, "%d.%m.%Y-%H:%M:%S",
&(p->start_time_tm))==NULL) {
......@@ -281,16 +306,44 @@ int check_conf(flow_spec_t **flow_spec) {
return 0;
} /* check_conf() */
void print_conf(flow_spec_t **flow_spec) {
void print_conf(global_spec_t *global_spec, flow_spec_t **flow_spec) {
flow_spec_t **pp;
flow_spec_t *p;
char **rr;
char *r;
int i, j;
if (flow_spec==NULL)
if (global_spec==NULL || flow_spec==NULL)
return;
/* Print global configuration */
if (global_spec->conf_file)
printf("global_spec->conf_file: %s\n", global_spec->conf_file);
else
printf("global_spec->conf_file: unspecified\n");
printf("global_spec->no_stdout: %d\n", global_spec->no_stdout);
if (global_spec->db)
printf("global_spec->db: %s\n", global_spec->db);
else
printf("global_spec->db: unspecified\n");
if (global_spec->user)
printf("global_spec->user: %s\n", global_spec->user);
else
printf("global_spec->user: unspecified\n");
if (global_spec->passwd)
printf("global_spec->passwd: %s\n", global_spec->passwd);
else
printf("global_spec->passwd: unspecified\n");
printf("global_spec->hostname" %s\n", global_spec->hostname);
/* Print specified flows */
pp=flow_spec;
p=*pp; i=0;
while (p) {
......
#ifndef __ABW_CONF_H
#define __ABW_CONF_H
#include "abw.h"
#define MAX_CHARACTERISTICS_STRING 128
......@@ -5,6 +8,8 @@
int read_conf_file(char *filename, flow_spec_t **flow_spec);
int check_conf(flow_spec_t **flow_spec);
int check_conf(global_spec_t *global_spec, flow_spec_t **flow_spec);
void print_conf(global_spec_t *global_spec, flow_spec_t **flow_spec);
void print_conf(flow_spec_t **flow_spec);
#endif
......@@ -245,11 +245,6 @@ int main(int argc, char *argv[])
}
}
/* if (!strlen(header_filter)) {
fprintf(stderr, "%s: header filter must not be empty\n", __func__);
exit(-1);
} */
printf("%s: header_filter: |%s|, device: |%s|, sau_mode: %c, sau_threshold: %u\n", __func__, header_filter, device, sau_mode, sau_threshold);
while (1) {
......@@ -257,19 +252,14 @@ int main(int argc, char *argv[])
fprintf(stderr, "%s: gettimeofday() failed\n", __func__);
exit(-1);
}
printf("1 %u.%u %d/%d %d/%d %d/%d %d/%d %d/%d\n",
(unsigned int)tm.tv_sec, (unsigned int)tm.tv_usec,
rand()%1000, rand()%1000, rand()%1000, rand()%1000,
rand()%1000, rand()%1000, rand()%1000, rand()%1000,
rand()%1000, rand()%1000);
printf("2 %u.%u %d/%d %d/%d %d/%d %d/%d %d/%d\n",
(unsigned int)tm.tv_sec, (unsigned int)tm.tv_usec,
rand()%1000, rand()%1000, rand()%1000, rand()%1000,
rand()%1000, rand()%1000, rand()%1000, rand()%1000,
rand()%1000, rand()%1000);
usleep(100000);
for (i=1; i<=2; i++) {
printf("%d %u.%u %d/%d %d/%d %d/%d %d/%d %d/%d\n",
i, (unsigned int)tm.tv_sec, (unsigned int)tm.tv_usec,
rand()%1000, rand()%1000, rand()%1000, rand()%1000,
rand()%1000, rand()%1000, rand()%1000, rand()%1000,
rand()%1000, rand()%1000);
}
usleep(100000);
}
return 0;
......
#ifndef __ABW_MYSQL_H
#define __ABW_MYSQL_H
#include "abw.h"
#define MAX_QUERY 256
int abw_mysql_get_subject_id(global_spec_t *global_spec,
......@@ -10,3 +15,5 @@ int abw_mysql_insert_value(global_spec_t *global_spec, flow_run_t *q,
int timestamp_sec, int timestamp_usec,
char *characteristics,
int value_int, double value_float, char *value_varchar);
#endif
#ifndef __ABW_TIME_H
#define __ABW_TIME_H
int abw_next_timestamp(struct timeval *interval,
struct timeval *next,
struct timeval *wait);
#endif
......@@ -16,9 +16,10 @@ DROP TABLE IF EXISTS tSubject;
CREATE TABLE tSubject (
id int NOT NULL auto_increment,
hostname varchar(64),
ip varchar(15),
ip varchar(39),
interface varchar(32),
direction varchar(3),
link_mbps int,
PRIMARY KEY (id)
);
......@@ -31,14 +32,14 @@ CREATE TABLE tParameters (
payload_string varchar(128),
interval_sec int,
interval_usec int,
link_mbps int,
PRIMARY KEY (id)
);
DROP TABLE IF EXISTS tValue;
CREATE TABLE tValue (
id int NOT NULL auto_increment,
measurement_id int NOT NULL,
subject_id int NOT NULL,
parameters_id int NOT NULL,
timestamp_sec int,
timestamp_usec int,
characteristics varchar(32),
......
<?php
define("CURRENT_VERSION", "1.22");
define("IPTA_HOME", "/usr/local/ipta");
/*
* Global variables
*/
$monthNumberToLongName[1]="January";
$monthNumberToLongName[2]="Februry";
$monthNumberToLongName[3]="March";
$monthNumberToLongName[4]="April";
$monthNumberToLongName[5]="May";
$monthNumberToLongName[6]="June";
$monthNumberToLongName[7]="July";
$monthNumberToLongName[8]="August";
$monthNumberToLongName[9]="September";
$monthNumberToLongName[10]="October";
$monthNumberToLongName[11]="November";
$monthNumberToLongName[12]="December";
$monthNumberToShortName[1]="Jan";
$monthNumberToShortName[2]="Feb";
$monthNumberToShortName[3]="Mar";
$monthNumberToShortName[4]="Apr";
$monthNumberToShortName[5]="May";
$monthNumberToShortName[6]="Jun";
$monthNumberToShortName[7]="Jul";
$monthNumberToShortName[8]="Aug";
$monthNumberToShortName[9]="Sep";
$monthNumberToShortName[10]="Oct";
$monthNumberToShortName[11]="Nov";
$monthNumberToShortName[12]="Dec";
$firstYear=2000;
$firstMonth=8;
$firstDay=1;
$firstTimestamp=mktime(0, 0, 0, $firstMonth, $firstDay, $firstYear);
$firstDateTime=strftime("%d.%m.%Y %H:%M:%S", $firstTimestamp);
$currentTimestamp=time();
$currentYear=date("Y", $currentTimestamp);
$currentMonth=date("m", $currentTimestamp);
$currentDay=date("d", $currentTimestamp);
$currentDayOfWeek=date("w", $currentTimestamp);
if ($currentDayOfWeek==0)
$currentDayOfWeek=7;
$currentHour=date("H", $currentTimestamp);
$currentMinute=date("i", $currentTimestamp);
$currentSecond=date("s", $currentTimestamp);
/*
* Common functions
*/
/*
* If global variable $debugLevel is non-zero, prints "$func: $mes<br>\n".
*
* Input:
* $func - name of function or script from which this function is called
* $mes - debugging message to be printed
*/
function debug($func, $mes)
{
if ($GLOBALS[$debugLevel])
echo "$func: $mes<br>\n";
} /* debug() */
/*
* Converts a three-letter English name of a month (e.g., "Jan") to number 1-12.
*
* Input:
* $monthName - name of month to be converted
*