1
|
- Gabriele Carcassi
- STAR Collaboration
- 6 May 2002 - Proposal
|
2
|
- STAR uses MySQL to keep track of data files (file catalog)
- There are already many projects concerning file catalogs
- MAGDA already uses MySQL to store file catalog information
- We will not concentrate on this aspect
|
3
|
- STAR reconstruction jobs need access to both data files and databases
|
4
|
- DB is being replicated through MySQL replication
- Master/slave approach
|
5
|
- Database replication is essential because:
- Increase data availability
- Allows locations to run jobs independently (in case of network
congestion)
- Correction to the DB are available to all locations
- In order to successfully execute jobs in a GRID environment, we need
database replication to be somehow integrated
|
6
|
- Aid DB administrator to manage complexity of database replication
- Tools to install a mirror using GRID technology
- Database catalogs and integration with file catalogs
- Integrate GRID authentication
|
7
|
- The more server you have, the more complicated is to manage the system
- Build a GUI that helps the database administrator (DBA) to have a
general picture
|
8
|
- Keep track of all the servers, and the relationship between each other.
- Help manage more sophisticated network topologies, for example:
|
9
|
- Configurations
- Aid the comparison between server characteristics (i.e. OS version,
MySQL version)
- Aid the comparison of the settings of the different database servers
- User management
- Create and delete users on a group of servers
|
10
|
- Consistency checks
- Compare row counts of different replicas
- Compare master log pointer with slaves
- Check slave connections to the master
- Evaluate the replication
- Monitor CPU and network activity of the servers to help decide if the
current number of servers is sufficient
|
11
|
- MySQL keeps the slave synchronized, but you have to manually copy the db
files during slave initialization
- When the DB is already in place, you might have to copy each file by
itself, since the total might exceed 2GB
- GRID technologies can be used to transfer the first copy of the database
from site to site
|
12
|
- New mirror
- Steps for mirror creation
- Install MySQL (manual?)
- Copy database files (through GRIDFtp)
- Configure master and slave
- Creating/deleting a database on an existing mirror
|
13
|
- Ease of use
- Integration with the previous GUI
- Hide as much details as possible and encourage good configuration
policies (i.e. create a user with suitable permissions to be used by
the slave connection)
|
14
|
- Jobs need to know to which database server to connect (for now, this is
done by XML files)
- In a GRID environment, jobs will contact a file catalog to determine the
location of the files to be used
- The scheme for the database catalog shouldn’t be different, and it
should be as connected as possible to the file catalog
|
15
|
- Database catalog
- A job should query the catalog to know to which database server to
connect
- If possible, file catalogs could be used directly. For example, instead
of a physical file location, the catalog could return the parameters
for the database connection (i.e. “mydb.star.bnl.gov:3301”)
- If no connection can be established, the job might ask the catalog for
an alternative server
|
16
|
- Administration
- The DBA should be able to establish the policies that decide to which
server a given job should connect (i.e. IP address, user groups)
- Integration with the GUI
- The GUI would use the catalog to keep track of the different servers
- The GUI would help the DBA to assign policies, and check that the
catalog information is not corrupted
|
17
|
- GRID authentication should be integrated to the whole scheme
- Connection between database servers
- Connection to the catalog
- Connection job-database server
|
18
|
- MySQL 4.0 will integrate SSL for both user connections and between
server connections
- MySQL can be instructed to accept only certificate authentication
- MySQL 4.0 is still in alpha and SSL is not GSI, but:
- Some issues should be already addressed
- A patch to enable GSI on SSH is available and might provide further
insights
|
19
|
- Integration with the GUI
- Just one authentication at the beginning of the application
- Same authentication used to authenticate to the servers
- User management
|