METADATA USAGE TENDENCIES IN LATIN
AMERICAN ELECTRONIC JOURNALS
Rolando Coto-Solano1; Helena Francke2; Saray Córdoba-González3.
Office of the Vice Dean of Research, University of Costa Rica
Ciudad Universitaria Rodrigo Facio, San José, Costa Rica
e-mail:
[email protected]
2 Swedish School of Library and Information Science, University of Borås,
SE-501 90 Borås, Sweden
e-mail:
[email protected]
3 Office of the Vice Dean of Research, University of Costa Rica
Ciudad Universitaria Rodrigo Facio, San José, Costa Rica
e-mail:
[email protected]
1
Abstract
The present study investigates the extent to which metadata tags are used in Latin
American electronic journals, and whether these journals in fact provide basic information (abstracts, keywords, etc.) that could be tagged as metadata. The authors also studied multilingualism in the marked-up information and in the basic
information, particularly the use of English (which can help bring the scientific
production of Latin America to a wider audience). In total, 45% of the journals had
metadata; the metatags keywords and description were the most commonly used.
The inclusion of structured metadata from the Dublin Core Metadata Element Set
in the journals was found to be very low, only 13%, and primarily existed in journals from Argentina, Costa Rica, and Brazil. The articles examined did not always
include abstracts and keywords (84% and 77% respectively), but in the articles that
did have them, English was frequently used (85% in abstracts and 91% in keywords). The <title> element was found to be used deficiently: Only 42% of full text
OA articles had their actual title in the <title> tag, which can potentially affect visibility in a search engine results. In sum, the road to marked-up metadata in all
journals is still long, and there are great inconsistencies in how metadata are employed and in their content. The authors conclude that there are signs that support
and efforts to increase awareness of how metadata can easily be included in a
journal’s web site may result in improved metadata and greater visibility.
Keywords: Metadata; Scholarly journals; Latin America; Open Access journals; Metadata usage patterns.
312
Rolando Coto, Helena Francke, Saray Córdoba
1. Introduction
Open Access (OA) in Latin America has emerged as an alternative strategy for
making research visible in response to the under-representation of the region’s research output in the major systems of scholarly article dissemination around the
world. These OA articles can be found through search engines, and are thus easily
available to the scientific and student communities. However, the retrieval capabilities of a harvesting system (or even of a regular web search) are highly dependent
on the quality of the information being marked up for the document. Challenges to
search engines or aggregators of metadata due to inconsistencies or inadequate
quality in the metadata have been reported repeatedly [1, 2, 3, 4, 5]. Even in a field
as apparently innocuous as the titles on the web page, good quality is essential.
The average user spends only a few seconds sifting through the results of a web
search [6], and providing an irrelevant title displayed among the search results
will cause the loss of an opportunity to connect the researcher to the journal’s information. At the same time, previous research has demonstrated that users retrieve information more efficiently when using metadata [7].
With its point of departure in a previous study of the characteristics of Latin
American OA journals [8] and a study of metadata occurrence in English, French,
and German language OA journals [5, 9], this paper further explores the occurrence and the contents of marked-up metadata in Latin American OA journals.
These journals (with a sample size of 123) have been drawn from the LATINDEX Regional System of Scientific Journals of Latin America, the Caribbean, Spain and
Portugal - electronic resources index [10]. In addition, the occurrence of visible metadata in the form of bibliographic data displayed on the page will be placed in relation to the marked-up metadata. In order to put the use of metadata in Latin
American journals into perspective, the frequency and estimated quality of metadata in this sample will be compared to metadata usage in journals published primarily in Europe and North America [5, 8, 11], to examine whether or not the
usage patterns diverge.
Latin American journals are primarily published in three languages: Spanish,
Portuguese, and English, and very often in combinations of these. However, the
provision of metadata is irregular at best [8], in particular such metadata as multilingual abstracts and keywords (with some journals still failing to provide basic
data such as keywords in any language, for example). The addition of such information is not only beneficial for the 330+ million speakers of Spanish, but also for
the global community of researchers that uses other languages (such as English) to
communicate among themselves, and who might benefit from the results of
science produced in Latin America. The regular addition of multilingual information is vital for the visibility of the region’s science, and for increased opportunities
Metadata Usage Tendencies in Latin American Electronic Journals
313
of scientific co-operation in our modern world.
An improvement in both (non marked-up) visible and marked-up metadata
could be forthcoming, given that attempts to encourage the inclusion of multilingual metadata are being made by such initiatives to promote Latin American OA
journals as LATINDEX, SciELO and RedALyC. An English translation of title, abstract, and keywords is mandatory in the journals included in SciELO [12], and encouraged among the RedALyC journals (using the OAI-PMH version 2.0) [13], and
LATINDEX journals [10] as part of the quality criteria of the system. The results
from this paper can give a better picture of the current situation and potential
needs to increase these efforts in the future.
The inclusion of abstracts and keywords in scholarly articles still varies quite a
bit depending on disciplinary practices. They are, for instance, more common in
the exact sciences and medicine than in the humanities (e.g. [9], p. 263). In a study
of scientific journals by Berkenkotter and Huckin [14], they found that abstacts began to appear in the journals in the 1950s, and became more common, as well as
longer and more informative over the years (p. 34). Charles Bazerman [15] has
noted that in the American Psychological Association’s Publication Manual (APA),
the 1950s was also when summaries gradually began to turn into what we today
consider to be an abstract (and placed at the beginning rather than the end of the
article).
Abstracts and keywords could by many definitions be considered metadata,
and the metadata concept predates the Web. The word metadata itself was coined
in 1969, when Myers used it to name a data exploration company, and appeared in
print for the first time on a brochure in 1973 [22]. In a Web context, marked-up metadata have been used in electronic publications from early on, and the discussion
about metadata has been particularly active since approximately 1996 when the
Dublin Core Metadata Initiative [16] emerged as an alternative to MARC. This
event enhanced the spread of marked-up metadata and nowadays, metadata is often put forth as the essence of the Semantic Web. Whereas metadata is often discussed in terms of more advanced protocols such as OAI-PMH and RDF, what we
focus on here are the more basic bibliographic data that may be marked up in an
HTML file, included in the document properties of a PDF file, or simply included
as visual metadata. Thus, we apply a broad understanding of what metadata are,
in line with the “data about data” view. The HTML elements <title> and <meta>
exist in the very early versions of HTML recommendations (cf. e.g. [17]). <meta>,
an optional element, is often used with server-directed information through the
http-equiv attribute; with various types of metadata directed at human readers,
such as keywords, description, and author; and with metadata provided by HTML
generators. However, it can also be used with other standards, such as the Dublin
Core elements.
314
Rolando Coto, Helena Francke, Saray Córdoba
Bruce and Hillmann [18] outline three strategies for how metadata can be
created as part of larger collections: through author self-submission; through human judgement in the classification process; and through “automated text-extraction techniques to pull metadata from a text corpus” [18]. These strategies are also
relevant in cases of single journals, even though they may be effected less strategically. Authors may provide MS Word or PDF files that are used as are, with metadata attached more or less consciously. Furthermore, authors are often encouraged
to include an abstract, keywords, and affiliation data when they submit an article.
At the same time, the decision to mark these metadata in (X)HTML is often done
by the journal’s editorial staff, who is responsible for any attempts to structure the
journal’s metadata. Presumably, when it comes to OA journals, automated textextraction techniques are seldom used to extract such data as authors, abstracts,
and keywords, but various (X)HTML editing programs automatically add metadata concerning for instance content type, generator, and robot instructions.
When metadata are included in the source rather than placed and distributed
in a separate file, they are less easily included into aggregators and other services
for shared metadata. Embedded metadata are, however, an important first step to
providing shareable metadata, and they offer information about the journal or article which may be used in various ways – both directly by users, and, if the metadata are reasonably structured, to be collected automatically and included into
aggregators. In the latter case, it is important that the metadata maintain a minimum level of quality. Authors concerned with the quality of shareable metadata
have pointed to consistency and coherence, conformance to expectations and standards, as well as contextual data as important factors contributing to metadata
quality [18, 19]. However, these issues have mainly been discussed with regard to
repositories and aggregation services, not individual journals.
Previous research has shown that in Latin American journals, metadata are not
commonly used. Cordoba and Coto [8] found that only 35% of the journals studied
used them at any level. However, the work done by SciELO and REDALyC may
change these figures in the future. In a questionnaire study with editors of Brazilian e-journals in 2003-2004, few of the editors were familiar with such metadata
standardisation initiatives as the Dublin Core Metadata Element Set and OAIPMH [20]. Approximately one third of the editors answered that they knew about
the Dublin Core Metadata Element Set, but no journal claimed to use it.
Rovira and Marcos [11] analyzed how metadata was used in 61 Library and Information Science (LIS) journals that were published OA. 59% of the sites they analyzed lacked metadata that could be useful for information retrieval and metadata
was included in no more than 13% of the articles. The authors studied three different levels of the site: the front page, the first 50 pages nearby the front page, and
the first and second articles in each journal that was analyzed.
Metadata Usage Tendencies in Latin American Electronic Journals
315
In a study of 265 international editor-managed OA journals, where approximately 75% of the journals were published in Western Europe, North America,
Australia or New Zealand, Francke [5, 9] showed that most journals publishing in
HTML contained a <title> element, and 90% of the journals in the study included
the <meta> element in some form. Most common was the use of <meta> elements
targeted at servers, using the attribute http-equiv (76.5-80.5% depending on journal level), whereas keywords, description, and generator occurred in one fourth to
one third of the journals in the study. Metadata marked up using one of the Dublin
Core elements were present in less than 10% of the journals. In that study, the
quality of metadata was generally fairly low – consistency in content between issues of the same journal was often lacking, and reliability and timeliness of the data was not always as high as could be wished for.
Even if metadata were not marked up in HTML, they more commonly occurred as visible metadata on the article page or on another place in the journal,
such as an article presentation page or in the table of contents. Thus, the name of
the author and the article title were present in all journals; the author’s affiliation
in 86.8%; abstracts in 78.9%; and keywords in 40.4% of the journals [5].
In light of these previous studies, we have looked at Latin American OA journals’ use of visible and marked-up metadata to ascertain if and where there may
be a particular need to draw journal editors’ attention to how they may further
improve the visibility of the articles published in their journals.
2.
Methodology
The present study investigates Latin American OA journals with a methodology
which draws on the quantitative survey in [8] and expands that study by a further
focus on metadata, particularly with regard to multilingualism. The sample has
been selected among the journals included in the LATINDEX database. The journals are peer reviewed, Open Access, and contain at least 40% scientific content.
They are published independently, not as part of a larger collection site (such as
SciELO of RedALyC) that might have caused their content to conform to very
similar publishing standards. We selected a sample from the 12 different countries
and territories that had more than 10 journals in the catalogue from the Electronic
Resources Index: Argentina, Brazil, Chile, Colombia, Costa Rica, Cuba, Ecuador,
Mexico, Peru, Puerto Rico, Uruguay and Venezuela. Within each country group,
we randomly chose 10% of the journals, which resulted in a sample of 167 titles.
When the bulletins, non peer-reviewed journals, and science education magazines had been discarded, together with the SciELO and RedALyC journals, we
316
Rolando Coto, Helena Francke, Saray Córdoba
reached 123 journals with an estimated sampling error of 7.5%. From the selected
journals, we chose the most recent issue and one of the research articles, most often the first one that was not an editorial, a book review, or other non-scientific article. As many international organizations divide Latin America into four subregions, we chose this partition to compare the data between them, as there are
differences in the publishing patterns in the various sub-regions. For instance,
Chile, Argentina and Brazil (in the Southern part of the continent) have publishing
cultures that are much more developed than those of the Andean Area. The subregions we use are: North: Mexico and Central America; Andean: Colombia, Venezuela, Peru, Bolivia, and Ecuador; South: Chile, Argentina, Uruguay, Paraguay,
and Brazil; Caribbean: The nations in the Caribbean Sea and Puerto Rico.
2.1. Survey study
We have studied four levels within individual journals:
(i) Cover: Cover page of the electronic journal (entry page, first page)
(ii) Table of contents (TOC): Table of contents of the latest issue
(iii) Article presentation page: Presentation page for an article, which contains information but not the “full text” of the article (if it exists).
(iv) Article text: Full text page for an article.
The focus of the study has been on the representation of metadata, both visible and
marked up. However, an important aspect, both when it comes to how the metadata may be used and the flexibility of what metadata can be included, is what file
format the journals – and particularly the articles – are published in. In Francke’s
study described above [9], cover and TOC level pages were mainly published in
some form of HTML, whereas the diversity was much larger at the article level.
Other factors that may influence the usability of metadata are the use of frames or
Content Management Systems, and the validity of the markup. The technical implication of frames is that what is perceived as one web page is made up by several
files. Each file may contain metadata, and at worst, the metadata are in conflict
with each other. Furthermore, it may not be advantageous with regard to retrieval
(e.g. in the results list of a search engine search) that all pages on the site have the
same value to the <title> element, namely that of the frameset file [5]. More and
more journals use some form of journal management system, such as the PKP’s
Open Journal System, which often provide ample opportunities for metadata inclusion. Thus, a comprehensive use of metadata may be prompted by such a system.
In terms of usefulness for retrieval or harvesting of metadata, harvesting of metadata may be easier to achieve if the HTML files are valid HTML. This was the motivation to also look at the validity of the HTML files at the various journal levels.
317
Metadata Usage Tendencies in Latin American Electronic Journals
Three types of metadata were studied. These were the existence of and value of
the <meta> element and the <title> element respectively (or the document properties for PDF files), and the inclusion of abstracts, keywords, author affiliation, and
date of submission/acceptance/publication for the articles. The languages of abstracts and keywords were noted specifically, because it is of importance to international visibility and accessibility which languages are used. The details of the
variables will be further elaborated in relation to the presentation of results below.
The data were processed using a Microsoft Access ® 2003 database with JETSQL. For sections 3.4 and 3.5, the inferential statistics were calculated using the
SAS JMP 8.0 statistical software.
3.
Results
In this section, we present the findings on metadata usage in the Latin American
OA journals included in the sample. The findings are discussed further under the
heading Discussion below.
3.1. General characteristics of the sample
Our corpus was made of 123 journals. From these journals, 81% of the latest issues
were published between 2007 and 2009. Only 3.3% were published prior to 2000.
In other words, as of April 2009, 60% of the sample was less than a year old and
75% was less than two years old. Two journals in the sample did not offer any way
of identifying their publishing date.
The journals used four different types of file formats for publishing their articles (Table 1).
Format
Journals
(Subformat)
Journals
HTML
33 (26.8%)
HTML 3.2
HTML 4.0 Transitional
HTML 4.01 Frameset
HTML 4.01 Transitional
HTML Non-specified
XHTML 1.0 Frameset
XHTML 1.0 Transitional
1 (0.8%)
3 (2.5%)
1 (0.8%)
13 (10.7%)
7 (5.8%)
2 (1.7%)
6 (5.0%)
PDF
105 (85.4%)
Both PDF and HTML
Other formats
17 (13.8%)
2 (1.6%)
Flash
MS-Word
1 (0.8%)
1 (0.8%)
Table 1. File formats in the sample.
318
Rolando Coto, Helena Francke, Saray Córdoba
85% of the journals published articles in PDF and 27% published their articles in
(X)HTML format. 14% of the journals (included in the previous figures) in fact
used both HTML and PDF for their articles. If we compare these data to corresponding data from 2008 [7], we find only small differences: In 2008, 83% of the
sample published the articles in PDF, 27% in HTML, and 11% of the journals used
both file formats.
Most of the journals use the native language of their country of publication in
their articles. Only 13% of the journals used English and approximately 3% had
articles in a third language.
Content Management Systems of some sort were used by 29% of the journals.
While most of the encountered examples were simply embedding the journal into
a pre-existing institutional platform (such as the Mexican e-Journal and the Cuban
and Brazilian BVS), 9% of the journals used OJS (particularly in Brazil and Argentina), and slightly more than 5% used other CMS systems, as is shown in Table 2.
Platform
Journals that use the platform
Total
Institutional Portals
OJS (OJS-PKP, SEER/OJS)
Joomla!
Lapacho
Drupal
36 (29.2%)
18 (14.6 %)
11 (8.9%)
4 (3.3%)
2 (1.6%)
1 (0.8%)
Table 2. Content Management Systems found.
As many as 31% of the journals use frames. This makes it even more important to
consider how and where marked-up metadata are included in order to be displayed and used efficiently. The use of frames could mean more work for the editors and fewer possibilities to find or share metadata.
3.2. Journal levels and validity
Electronic journals can have a variety of levels, depending on what choices have
been made with regard to their architecture. Not least, subscription journals may
require quite a few levels to be visited before the reader reaches the article file.
Most OA journals include at least a cover page, a table of contents, and an article
level. As described in Table 3, all journals but one in the sample have cover pages.
The fact that one journal is missing a cover page is in itself surprising, given that
one would expect that all journals have cover pages of some sort. The one journal
that lacked a cover page provided only a list of files on an FTP frame, which could
not be considered as a “cover”. While many journals have tables of contents pages,
fewer than 30% have article presentation pages (a practice common in other pub-
319
Metadata Usage Tendencies in Latin American Electronic Journals
lishing cultures), and only about 27% have full text articles in HTML format. The
article presentation pages could serve a particularly important role when the article file is offered in a format other than HTML, both in terms of providing a place
to include marked-up metadata and as a source of information for those who may
for some reason not have the proper software installed to view PDF files or Flash
pages, for instance.
Total
journals
Have cover
page
Have table of
contents
Have article
presentation page
Have article in
full text (HTML)
123
122 (99.2%)
100 (81.3%)
35 (28.5%)
33 (26.8%)
Table 3. Levels of the journals.
That the HTML pages of a journal are valid HTML can be of importance in terms
of offering long-term accessibility to the files, as well as to make it easier to automatically retrieve marked-up or visual metadata from a file (cf. [5]).For this reason,
the validation errors of the files at each journal level were investigated and are reported in Table 4. The figures are sometimes higher than would have resulted
from a manual investigation of the files, as the figures were generated by the W3C
HTML Validator, where one mistake in the coding may result in several errors. As
can be seen from these figures, there are journals with valid HTML files, but there
are great differences between journals.
Validation
errors
Median
Highest value
Lowest value
Cover
page
Table of
contents
27
584
1
Article presentation
page
27
787
0
13
292
0
Article full text
page (HTML)
18,5
1257
0
Table 4. Validation errors.
3.3. Marked-up metatags (<meta>)
Table 5 describes the usage of metatags within the journals in this study. While at
first glance many journals appear to use metadata (87% use some form of metadata in their cover page, for example), most of these metadata are computer generated, and of very little utility to describe the actual contents of the web page. When
we eliminate the computer generated http-equiv tags <Content-Type> and <generator>, and the automatically generated <ProgID>, a radically different picture
emerges. Only 45% of the journals have metadata in their cover page. The percentage rises at the article presentation page and full text page levels if we look at the
320
Rolando Coto, Helena Francke, Saray Córdoba
existence of <meta> tags in relation to the total number of journals with HTML
files at these levels (57% and 49% respectively for non-automatic metadata).
Cover
page
Have any kind of metadata
Have non-automatic metadata
(n = 122)
Table
of
contents
(n = 100)
Article
presentation
page
(n = 35)
Article full
text page
(HTML)
(n = 33)
107 (87.7%)
55 (45.1%)
84 (84.0%)
42 (42.0%)
34 (97.1%)
20 (57.1%)
25 (75.8%)
16 (48.5%)
Table 5. Use of metadata in the journals.
Tables 6a and 6b list the most used tags within each level of the journals. While
they vary for each level, the meta description (which is highly priced by search engines)
is the only one that appears among the most common attributes on every level.
Level: Cover (n = 55)
<meta> attribute
Occurrence
keywords
description
author
robots
copyright
DC.Creator
revisit-after
Content-Language
32 (58.2%)
32 (58.2%)
15 (27.3%)
14 (25.5%)
6 (10.9%)
6 (10.9%)
6 (10.9%)
5 (9.1%)
Level: Table of contents (n = 42)
<meta> attribute
Ocurrence
keywords
description
robots
author
DC.Creator
revisit-after
Content-Language
copyright
25 (59.5%)
24 (57.1%)
13 (31.0%)
11 (26.2%)
6 (14.3%)
5 (11.9%)
5 (11.9%)
5 (11.9%)
Table 6a. Most common metatags on the “cover page” and “table of contents” levels.
Level: Article presentation page (n = 20)
<meta> attribute
Occurrence
DC.Language
DC.Title
DC.Description
DC.Type
DC.Subject
DC.Creator
DC.Identifier
description
10 (50.0%)
10 (50.0%)
9 (45.0%)
9 (45.0%)
8 (40.0%)
8 (40.0%)
8 (40.0%)
8 (40.0%)
Level: Article full text page (n = 16)
<meta> attribute
Ocurrence
keywords
description
author
robots
Originator
pragma
expires
Content-Language
8 (50.0%)
8 (50.0%)
4 (25.0%)
3 (18.8%)
3 (18.8%
2 (12.5%)
2 (12.5%)
2 (12.5%)
Table 6b.Most common metatags on the “article presentation page” and the “article full
text” levels.
Table 7 describes the use of metadata within the PDF files. While 105 journals use
PDF to publish their articles, only 65 of those PDFs had any information within
321
Metadata Usage Tendencies in Latin American Electronic Journals
their metadata spaces. The situation looks quite promising in this table. However,
the way the metadata are worded indicates that these metatags are almost never
considered by the editors, and the tags almost always contain information that has
little or no relevance to the article.
PDF (n = 65)
Metadata type
Occurence
Has at least one word in
common with the actual datum
Matches the
actual datum
Title
63 (96.9%)
15 (23.8%)
4 (6,3%)
Author
49 (75.4%)
7 (14.3%)
5 (10,2%)
Subject
9 (13.8%)
-
-
Keywords
3 (4.6%)
3 (100%)
3 (100%)
Table 7. Metadata in PDF files.
3.4. Dublin Core Metatags
Among all journals, only 13% use Dublin Core metatags. Table 8 describes the use
of DC tags by region, and table 9 describes it by discipline. Whereas no specific
group stands out in the discipline table, the region table suggests a pattern of usage, pointing to more use of DC metatags in South and Central/North America.
The region grouping is not significant (p = 0.11). However, if we separate the
groups into their constituent countries, a significant pattern emerges. Table 9 describes the use of DC tags by country and clearly sets apart Costa Rica and Argentina from the rest of the countries and territories. This might be due to the fact that
both of these countries have constant training programs, which have aimed,
among other things, to raise awareness about standards such as Dublin Core metatags. While Brazilian journals report a use of 17% of DC tags (which is high within
this group of countries), Brazil does not yet come as significantly different from the
lowest signification group, which is surprising given that it also has important
programs to provide the editors with pre-installed CMS platforms. These observations will be further reviewed in the discussion section.
Region
All regions
Andean
Caribbean
Central / North
South
Total journals
123
23
10
34
56
Journals with any kind of DC tags, on any level
16 (13.0%)
1 (4.3%)
0 (0.0%)
5 (14.7%)
10 (17.9%)
Table 8. Dublin core metatag usage by region.
322
Rolando Coto, Helena Francke, Saray Córdoba
Argentina
Brazil
Chile
Colombia
Costa Rica
Cuba
Ecuador
Mexico
Peru
Puerto Rico
Uruguay
Venezuela
Total journals
% of usage of
DC
Signification groups
(α = 0,05)
17
23
12
11
6
6
2
28
3
4
4
7
35.3%
17.4%
0.0%
9.1%
66.7%
0.0%
0.0%
3.5%
0.0%
0.0%
0.0%
0.0%
B
BC
C
C
A
C
BC
C
BC
C
C
C
Table 9. Usage of Dublin Core elements by country. Groups not connected by the same
letter are significantly different.
3.5. Presence of basic information describing the article (abstract, keywords,
etc.)
Table 10 describes the occurrence of features considered basic when publishing a
scientific article. These features should ideally be present in all articles, as they
help authors, readers, and information specialists to catalog and effectively distribute and retrieve the results reported in the article, as well as in deciding if the
article is of interest to the reader. However, only 84% of the articles had abstracts,
and only 77% of them had keywords. As can be seen, the occurrence of author affiliation information is not much higher. The 45% occurrence of “date of reception/acceptance” should be considered an urgent task to tackle by the editors.
Whereas more than three quarters of the journals included keywords and abstracts in their visual metadata, and more than 60% of the journals have keywords
and abstracts in more than one language, these data are seldom marked up as metadata. The title was the information that was most commonly marked up within
the metatags, but this was the case in only 14% of the articles. Multilingual marking in the metas was extremely uncommon.
As for the occurrence of abstracts and keywords across disciplines, we found
that there was a tendency (p < 0.1) for Engineering and Medical journals to more
often include abstracts and keywords than the rest of the disciplines (the full comparison is presented in table 11). This can be connected to varying publication
practices in different epistemic cultures (cf. [21, 9]). However, it is interesting that
abstracts and keywords are relatively rare in the exact and natural science journals
in this study, as the publication practices in these cultures would lead us to expect
323
Metadata Usage Tendencies in Latin American Electronic Journals
that they would include abstracts and keywords more often than journals in the
social sciences and humanities.
Article features
(n=123)
Articles with
the feature
Multilingual
Marked as
<meta>
Two or more languages marked as <meta>
Title
123 (100%)
38 (31.0%)
17 (13.8%)
1 (0.8%)
Abstract
103 (83.7%)
80 (65.0%)
9 (7.3%)
2 (1.6%)
Keywords
95 (77.2%)
78 (63.4%)
8 (6.5%)
1 (0.8%)
Author affiliation
105 (85.4%)
-
5 (4.1%)
-
Date received /
accepted
55 (44.7%)
-
4 (3.3%)
-
Table 10. Occurrence of basic article features (titles, abstracts, keywords, author affiliation
and date of reception/acceptance)
Subject
Journals
with
abstracts
(n = 103)
Signification
groups
Signification
groups
( α = 0.1 )
Journals
with
keywords
(n = 95)
Engineering sciences
Medical sciences
Agricultural sciences
Multidisciplinary
Social Sciences
Arts and Humanities
Exact and Natural Sciences
8 (100.0%)
24 (96.0%)
7 (87.5%)
5 (83.3%)
44 (80.4%)
8 (72.7%)
7 (70.0%)
AB
A
ABC
ABC
BC
C
C
8 (100.0%)
23 (92.0%)
7 (87.5%)
4 (66.7%)
39 (71.4%)
7 (63.6%)
7 (70.0%)
A
A
AB
AB
B
B
AB
( α = 0.1 )
Table 11. Use of abstracts and keywords in Latin American Journals. Groups not connected
by the same letter are significantly different.
Article feature
Title
Total articles
123
Abstract Keywords
103
95
Use of one language
National language only
English Only
Third Language only
66 (53.7%) 13 (12.6%) 7 (7.4%)
16 (13.0%) 10 (9.7%) 10 (10.5%)
3 (2.4%) 0 (0.0%)
0 (0.0%)
Use of two or more languages
Both national language and English
Both Spanish and Portuguese
National language, English and a third language
33 (26.8%) 69 (67.0%) 70 (73.7%)
0 (0.0%) 2 (1.9%)
1 (1.1%)
5 (4.1%) 9 (8.7%)
7 (7.4%)
Table 12. Languages used in the article features studied. Percentages have been calculated
based on the number of journals in which the feature (title, abstract, keywords)
exists.
324
Rolando Coto, Helena Francke, Saray Córdoba
Of the articles that had abstracts, about 15% of them did not include a version of
the abstract in English. (In all cases where a “third language” was found, the article also contained the information in both English and the national language). The
translation of the titles, however, was much less frequent: only about 46% of the
articles had a version of their titles in English.
3.6. Titles
It is alarming in itself that only 76% of the journals have their own title marked up
as <title> on the cover page; even more so that the title of the article occurs so infrequently in the presentation pages (29%) and in the full text pages (42%). This is
particularly distressing when we take into account that one of the premises of
Open Access is web access. The titles of the web pages are the most prominent part
of the search results displayed by all major search engines presently in use, and the
fact that more than half of the web pages examined do not even feature the name
of the article (and that quite a few contain elements that have nothing to do with
the title) weighs heavily on their capability to be retrieved, and ultimately, on their
visibility.
<title> content
Level: Cover
(n = 122)
Level: TOC
(n = 100)
Journal title
93 (76.2%)
71 (71.0%)
Institution name
24 (19.7%)
17 (17.0%)
Issue information
2 (1.6%)
49 (49.0%)
Two or more of the above
None of the above
9 (7.4%)
10 (8.2%)
41 (41.0%)
10 (10.0%)
Table 13a. Title elements (levels “cover” and “table of contents”).
<title> content
Level: Presentation
page (n = 35)
Level: Article full text
page (HTML) (n = 33)
Journal title
Institution name
Issue information
Two or more of the above
14 (40.0%)
12 (34.3%)
6 (17.1%)
7 (20.0%)
14 (42.4%)
12 (36.4%)
5 (15.2%)
4 (12.1%)
Article title
Author name
Both article title and author’s name
None of the above
10 (28.6%)
8 (22.9%)
0 (0.0%)
3 (8.6%)
14 (42.4%)
4 (12.1%)
0 (0.0%)
1 (3.0%)
Table 13b. Title elements (levels “presentation page” and “full text HTML page”)
Metadata Usage Tendencies in Latin American Electronic Journals
325
Titles should be unique identifiers of each page, and as such, they should be
unique for every page. However, of all journals, 49% had titles repeated on two or
more levels, and 31% had the exact same title on all levels.
4.
Discussion
The first thing to be noted about these results is the widespread use of PDF files to
publish the journals, almost never accompanied by their corresponding HTML
versions. This use is even more common among the journals in this sample than
was the case in the study of English, French, and German language journals reported in [9]. In Francke’s study [9, p. 209-211], there were signs of a trend from
HTML publishing of articles to PDF publishing when the choice of file format of
the articles in the first issues of the journals was compared to that in the most recent issues. The prevalence of PDF in the Latin American journals could thus be
either a regional pattern or could be part of an international trend towards more
and more PDF publishing in open access journals, or could be a response to the
relative ease of creating PDF files. Compared to the journals in Francke’s study,
the number of different file formats is also lower, illustrating what might be a
trend towards PDF and HTML publishing. However, the percentage of journals
that publish articles in another format than either of the two dominating ones is
equivalent in the two studies.
If there is indeed a move towards PDF publishing, as these two studies indicate, it will have consequences for the use of metadata as it is more difficult to extract metadata from PDF files than from HTML files, and as this study also shows
signs that editors much more seldom pay attention to the content of metadata entries in PDF files than they do in HTML files. It is not uncommon that the title information contains the name of the file that was used to generate the PDF file, and
that the author field gives the name of the person registered to the original software. Table 7 shows that subject and keywords are seldom included in the PDF
files. One way in which to handle the article-level metadata is to place them in the
<meta> element of an article presentation page in HTML. These pages are in fact
(very) slightly more common than article files in HTML in the present study.
The most commonly used metatag among the journals in the sample is keywords, which roughly coincides with the findings of Rovira & Marcos [11]. While
the tag keywords has been discredited because of black hat search engine optimization practices such as keyword stuffing, the second most common tag, description,
is still immensely important (Google, for example, frequently uses it as the snippet
to display below the web page’s title). The figures for the meta attributes key-
326
Rolando Coto, Helena Francke, Saray Córdoba
words, description, and author are in all cases higher in the study of English,
French, and German language journals [9] than in the present study, but it is only
a case of fairly small differences except in the case of keywords and descriptions
on the cover page and author information in the article files, which are decidedly
more common among the journals in the earlier study. On the other hand, the
attribute robots occurs more frequently among the Latin American journals, and
metadata that are marked up with Dublin Core elements are more frequent in the
Latin American study.
Even though Dublin Core is still rarely used, the pattern of use of DC metatags
shown in tables 8a and 8b point in an interesting direction. The two countries that
are significantly different in the sample (Costa Rica and Argentina) have tried to
maintain regular efforts to train editors at a local level, and thus raise awareness
about metadata standards through initiatives such as the LATINDEX Project in
Costa Rica and the CAICYT Office in Argentina. Overall, it could be argued that
training (in the form of workshops and production of manuals), is a way to promote the use of higher-quality metadata, and further studies could help clarify and
confirm these findings.
On a final note about the Dublin Core norms, their use is slightly higher in the
Southern region; probably because of its more developed editorial cultures and
higher presence in indexes such as the ISI (this is particularly true of Brazil, Chile
and Argentina).
Returning to metadata in general, the study from 2008 [8] showed that 35% of
the journals used some kind of non-automatically marked-up metadata. There has
been an improvement in this number, which is now reported as 45%. This increase
will have to be verified in future years to determine whether the trend towards increased use of metadata continues in the region.
Based on this sample, it does not seem as though Latin American OA journals
are provided with marked-up metadata through the <meta> element more seldom
than journals from other parts of the world. Rather, the use reflects the fairly low
occurrence of the <meta> element in many independent OA journals in other areas.
Francke [5] has argued that this low use could be a reflection of the lack of routines
for assigning metadata to resources prior to publication. In Latin America, this pitfall is made worse by a lack of knowledge of these standards, and by the additional costs (not so much in equipment and software, as for training of human
personnel) involved in the manual manipulation of metadata.
One of the most prominent results from the study is the low use of nonnational languages in the titles (46%) and as the main language of the articles
(16%). A very interesting part of the reality in the region is the ongoing debate of
the national languages as “languages of science”, as opposed to the use of English
as a vehicle of international scientific communication. This is particularly true in
Metadata Usage Tendencies in Latin American Electronic Journals
327
the case of Spanish. The efforts of official authorities such as the Spanish Real Academia de la Lengua, combined with a strong linguistic self-esteem, the sheer number
of speakers, and the feeling that “local science should be communicated to the locals, in their language”, have lead to comparatively smaller use of English as an
academic language in Latin America. This clashes directly with the policy of systems such as Scopus, that demand their journals to be written in English. Systems
such as SciELO and RedALyC have progressively become more strict in demanding that articles have abstracts and keywords in English (LATINDEX merely recommends that they do), which might contribute to the increased use of English in
the journals examined (73% of all journals had abstracts in English; 72% had keywords in English).
The basic features that are described as basic information included as visual
metadata in or in relation to the articles, such as title, abstract, keywords, and author affiliation occur in more than 75% of these journals, similar to what Francke
[9] found in her study. The date when the article was received or accepted, or alternatively when the page was updated, occur less often, however, in less than half
of the journals in both studies. One remarkable difference between the findings of
the two studies is that keywords are significantly more common in the Latin
American journals. Possibly this could be a result of the fact that the number of
journals in the present study that come from the exact sciences and medicine is
larger than in the other study, which had a larger section of journals from the arts
and humanities. Abstracts and keywords are less common in the arts and humanities than in other academic areas, but this does not account for the smaller difference in the figures for abstracts in the two studies, even though fewer journals in
Francke’s study also included abstracts than was the case here.
Finally, even though the advantages of Content Management Systems geared
towards electronic publishing (such as OJS or EPrints) in terms of both time and
money have been documented, few journals in Latin America make use of these
platforms. Only about 29% of the journals use any kind of auxiliary platform, and
even fewer use free, open source programs to help their publishing processes. Brazil
is an important exception to this. More than 500 journals are already using the OJS
system in Brazil (more specifically, a locally-produced derivate of OJS named
SEES) [25]. The OJS system uses Dublin Core by default to describe the uploaded
content, so it potentially could help improve the quality of metadata in these journals.
The data in the present study show that even though Latin American OA journals are most often on par with and sometimes even better than journals from a
wider selection of countries at including marked-up and visual metadata on various levels of the journal, much work still remains in order for all journals to provide
standardized and high quality metadata. As the results from the study of the <title>
and <meta> elements prove, there is still a lack of consistency in which <meta>
328
Rolando Coto, Helena Francke, Saray Córdoba
elements are included and in what the <title> elements contain. There is also a low
use of established standards such as the Dublin Core Metadata Set. There is still
some way to go before the metadata can be considered shareable across journals.
5.
Conclusions
All journal editors face difficult questions when trying to balance their annual
budgets, and while many of them would like to invest in building top-of-the-line
web sites, fully optimized for search engines and with powerful options for the
readers, this remains a budgetary dream for most, something that in many cases is
put off as “maybe next year”. In Latin America most journals work as teams of a
few dedicated individuals, borrowing office space, computer power and assistant
time from their own research laboratories. These editors, toiling quietly in the
back, would be more than willing to incorporate new technologies into their existing journals, as long as the cost is not prohibitive and they have support from
someone who will explain to them just how to do it.
Let us consider, for example, what could be thought of as the cheapest of all the
technologies discussed here: setting a <title> tag for a web page. This requires
nothing more than a few keystrokes, a person who has read the title of the article,
and a naming standard for each level, which would not take more than a few minutes to define in an editorial board meeting. These simple measures take on a
new meaning when discussed against the background of search engine mechanisms and visibility. Google, for example, explicitly asks for programmers to make
<title> tags “descriptive and accurate” [23], while search engine optimization literature frequently quotes <title> as a vital part of ranking algorithms, and insists
that practices such as title repetitions could be penalized because they could potentially represent “duplicate content” [24]. If more information trickled down to
the editors that would explain to them why these considerations are important,
many of them would be willing to improve on the low rates of inclusion of metadata in our sample (only 42% of the articles had their own title in the <title> element). We believe that it is the role of the researchers to assist in the transfer of this
knowledge, and make sure it expands beyond the academic surrounding and into
the hands of editors and journal directors.
While PDFs are a useful resource, and still one of the most inexpensive ways to
publish electronic materials, the meta-information within the PDFs in this sample
was used erratically, and the document-centric view that focuses on PDFs distracts
editors from the fact that a structured web site (with table of contents and article
Metadata Usage Tendencies in Latin American Electronic Journals
329
presentation pages) provides very good opportunities to describe the contents of
their journal. As Francke [5] suggested, the use of a cover page and a ‘table of contents’ is not enough, and the use of article Presentation pages would greatly contribute to article retrievability in general when PDF is used as the publishing
format for the articles. Our results show that most of the keywords are concentrated on the cover and table of contents levels. The article presentation and article
full text pages, when they exist, do not use as many metadata, and are therefore
under-described. Since it is in these pages the information researchers are looking
for resides (information which is also sought after by aggregators and metadata
harvesters), more efforts are needed to optimize these pages and make them as
visible and well documented as possible.
Full text web pages (be they programmed using HTML, PHP, or XML displayed through CSS or XSLT templates) can take advantage of the many opportunities of interactivity, reader-input, and multimedia that are desirable for our
future scientific papers. Obviously the best of all solutions would be to provide
fully marked XML files for every article, which could be exploited for information
by numerous systems. This is precisely the promise of the semantic web when
combined with Open Science: Massive inter-operability of scientific information
and corpora, which could bring new ways of data mining, and ultimately of getting to information. The use of more data-centric programming techniques will
help our journals prepare for the Web 3.0 and its exciting possibilities. However,
even if XML marking could potentially render the <meta> tags obsolete because
any of the information currently thought of as a meta (abstracts, keywords, dates
of acceptance, authors and their information, etc.) would be properly marked, this
would bring us back to where we started: This information cannot be marked unless it exists. In Latin America most of the journals are published by small, independent units, not by large publishing companies. This involves a group of editors
from different epistemic cultures, many of them self-taught, that are not always
aware of why this information is important.
The cost of producing full XML articles is, at least for the time being, prohibitive to most of the publishing institutions in Latin America. (Only RedALyC and
SciELO, the two largest indexation systems in the region, have been able to provide extensive XML marking, thanks in part to public funding, in the first case
from the Mexican CONACYT National Council for Science and Technology and in the
second case by the Brazilian CNPq National Council for Research). But even if adequate funding for every journal is not available, we must not refrain from fixing
what we can now. The authors believe that training, particularly to explain how to
correct common mistakes, could have an impact on the visibility and retrievability
(both by humans and by automatic agents) of Latin American journals in the short
term. This will help bring the region’s science into the spotlight, and fulfill the
330
Rolando Coto, Helena Francke, Saray Córdoba
promise of Open Access to more students and researchers, no matter where in the
world they might be.
Acknowledgements
We would like to thank the organizers of ELPUB 2009, as well as the three anonymous reviewers for their valuable comments. The Costa Rican authors would like
to thank the MICIT (Costa Rican Ministry of Science and Technology) and the
University of Costa Rica for their support.
References
[1]
[2]
[3]
[4]
[5]
[6]
DUSHAY, N. and HILLMANN, D.I. Analyzing Metadata for Effective Use
and Re–use. In DC–2003: Proceedings of the International DCMI Metadata Conference and Workshop [online]. pp. 161–170. Available from: <http://www.side
rean.com/dc2003/501_Paper24.pdf> [cited 14 January 2009].
HUTT, A. and RILEY, J. Semantics and Syntax of Dublin Core Usage in Open
Archives Initiative Data Providers of Cultural Heritage Materials. In Proceedings of the 5th ACM/IEEE–CS Joint Conference on Digital Libraries, Denver, Colorado, June 7–11 June, 2005. New York: ACM Press, 2005. pp. 262–270.
SHREEVES, S.L., RILEY, J. and MILEWICZ, L. Moving towards Shareable
Metadata. First Monday [online]. 2006, vol. 11, no 8. Available from: <http://
firstmonday.org/issues/issue11_8/shreeves/index.html> [cited 14 January 2009].
ROBERTSON, R.J. and DAWSON, A. An Easy Option? OAI Static Repositories as a Method of Exposing Publishers’ Metadata to the Wider Information
Environment. In MARTENS, B; DOBREVA, M. ELPUB2006: Digital Spectrum:
Integrating Technology and Culture - Proceedings of the 10th International Conference on Electronic Publishing held in Bansko, Bulgaria 14-16 June 2006 [online].
pp. 59-70. Available from: <http://elpub.scix.net/data/works/att/261_elpub
2006.content.pdf> [cited 12 January 2008].
FRANCKE, H. The State of Metadata in Open Access Journals: Possibilities
and Restrictions. In CHAN, L.; MORNATI, S. ELPUB2008: Open Scholarship:
Authority, Community, and Sustainability in the Age of Web 2.0 - Proceedings of the
12th International Conference on Electronic Publishing held in Toronto, Canada 2527 June 2008 [online]. pp. 56-67. Available from: <http://elpub.scix.net/data/
works/att/056_elpub2008.content.pdf> [cited 12 January 2009].
AMAT, C.B. Rendimiento de 8 sistemas de recuperación de información del
espacio web español. El Profesional de la Información [online]. 2005, vol. 14,
no. 1. Available from: <http://eprints.rclis.org/7639/1/EPI.pdf> [cited 14 January
2009].
Metadata Usage Tendencies in Latin American Electronic Journals
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
331
SINGH, P., KLOBAS, J.E. and ANDERSON, K. Information Seeking Behaviour
of Electronic Records Management Systems (ERMS) Users: Implications
for Records Management Practices. Human IT [online]. 2007, vol. 9, no. 1.
pp. 135-181. Available from: <http://www.hb.se/bhs/ith/1-9/psjkka.pdf> [cited
14 January 2009].
CÓRDOBA-GONZÁLEZ, S. and COTO-SOLANO, R. Characteristics Shared
by the Scientific Electronic Journals of Latin America and the Caribbean. In
CHAN, L. and MORNATI, S. ELPUB2008: Open Scholarship: Authority, Community, and Sustainability in the Age of Web 2.0 - Proceedings of the 12th International Conference on Electronic Publishing held in Toronto, Canada 25-27 June 2008
[online]. pp. 187-202. Available from: <http://elpub.scix.net/data/works/att/
187_elpub2008.content.pdf> [cited 12 January 2009].
FRANCKE, H. (Re)creations of Scholarly Journals: Document and Information Arcitecture in Open Access Journals. Borås, Sweden: Valfrid, 2008. Also available
from: <http://hdl.handle.net/2320/1815/> [cited 12 January 2009].
LATINDEX. Documentos. Evaluación de publicaciones electrónicas [online].
2008. Available from: <http://www.latindex.org/latindex/busquedas1/revistas
_elec.html> [cited 13 January 2009].
ROVIRA, C. and MARCOS, M. Metadatos en revistas-e de Documentación de
libre acceso. El Profesional de la información. 2006, vol. 15, no. 2. Also available
from: <http://eprints.rclis.org/9452/1/doc.1.pdf> [cited 13 January 2009].
SciELO. SciELO Criteria: Criteria, Policy and Procedures for Admission and
Permanence of Scientific Journals in the SciELO Collection [online]. 2004.
Available from: <http://www.scielo.org/php/level.php?lang=en&component=
42&item=2> [cited 14 January 2009].
RedALyC. Inclusión de revistas [online]. 2008. Available from: <http://redalyc.
uaemex.mx/redalyc/src/proyecto/criterios.html> [cited 14 January 2009].
BERKENKOTTER, C. and HUCKIN, T.N. Genre Knowledge in Disciplinary Communication: Cognition/Culture/Power. Hillsdale, NJ: Lawrence Erlbaum, 1995.
BAZERMAN, C. Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science. Madison, WI: University of Wisconsin Press, 1988.
Also available from: <http://wac.colostate.edu/books/bazerman_shaping/> [9
April 2009].
DUBLIN CORE METADATA INITIATIVE [online]. 2009. Available from:
<http://dublincore.org/> [cited 14 January 2009].
BERNERS-LEE, T. and CONNOLLY, D. RFC 1866 - Hypertext Markup Language - 2.0 [online]. Network Working Group, 1995. Available from: <http://
tools.ietf.org/html/rfc1866> [cited 5 April 2009].
BRUCE, T.R. and HILLMANN. D.I. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In: HILLMANN, D.I. and WESTBROOKS, E.
332
Rolando Coto, Helena Francke, Saray Córdoba
Metadata in Practice. Chicago: ALA Editions, 2004. pp. 238-256.
[19] BARTON, J., CURRIER, S. and HEY, J.M.N. Building Quality Assurance into
Metadata Creation: An Analysis Based on Learning Objects and e-Prints
Communities of Practice. In: SUTTON, S., GREENBERG, J. and TENNIS, J.
Proceedings 2003 Dublin Core Conference: Suporting Communities of Discourse and
Practice - Metadata Research and Applications, Seattle, Washington (USA) [online].
Available from: http://eprints.erpanet.org/83/01/paper60.pdf> [cited 5 April
2009].
[20] MARCONDES, C.H., SAYÃO, L.F., MAIA, C.M.R., DANTAS, M.A.R. and
FARIA, W.S. State-of-the-art of Brazilian E-journals in Science and Technology. In ENGELEN, J; COSTA, S.M.S.; MOREIRA, A.C.S. 8th International Conference on Electronic Publishing, Brasilia, D.F., Brazil 23-26 June 2004 [online].
pp. 79-90. Available from: <http://elpub.scix.net/data/works/att/079elpub2004.
content.pdf> [cited 5 April 2009].
[21] KNORR CETINA, K. Epistemic Cultures: How the Sciences Make Knowledge.
Cambridge, MA & London: Harvard University Press, 1999.
[22] MÉNDEZ RODRÍGUEZ, E. Metadatos y recuperación de información:
Estándares, problemas y aplicabilidad en bibliotecas digitales. Gijón, Spain:
Ediciones Trea, S.L., 2002.
[23] GOOGLE CORPORATION. Webmaster Guidelines. Available from: <http://
www.google.com/support/webmasters/bin/answer.py?answer=35769#1> [cited
on 5 April 2009].
[24] SIROVICH, J. and DARIE, C. Professional Search Engine Optimization with
PHP. Indiana: Wiley Publishing, Inc., 2007.
[25] IBICT. Notícias: Acesso livre à informação científica impulsiona desenvolvimento do País (17.02.09). Available from: <http://www.ibict.br/noticia.php?
id=596> [cited on 5 april 2009].
ANNEX I
8. ADDENDUM 1: NAMES OF EXAMINED JOURNALS
(TOTAL JOURNALS = 123)
Argentina
AdVersus
Archivos argentinos de alergia e
inmunología clínica
Biocell
Dermatología Argentina
Equipo Federal del Trabajo
Foro Iberoamericano sobre Estrategias
de Comunicación
Hologramática
Journal of Applied Economics
Journal of Computer Science and
Technology
Psikeba
Rev. Argentina de Lingüística
Rev. De Investigaciones Agropecuarias
Revista de Ciencias Sociales
Revista de Economía Política de Buenos
Aires
Telondefondo
Universitas
Urbe et Ius
Brazil
Afro Asia
Boletim do Instituto de Pesca
Brazilian Administration Review
Brazilian Journal of Biomotricity
Caderno espaço feminino
Caderno Virtual de Turismo
Contingentia
Data Grama Zero
Economia e Energia
Educação Temática Digital
Engenharia Ambiental
Hegemonia
Klepsidra
Online Brazilian Journal of Nursing
Relações públicas em revista
Revista brasileira de educação médica
(Online)
Revista Brasileira de Zoologia
Revista de Estudos da Religião
Revista de Gestão da Tecnologia e
Sistemas de Informação
Revista Eletrônica de Estudos
Hegelianos
Revista Expectativa
Revista Matéria
Semina
Chile
Agenda Pública
Ciencia y Trabajo
Cinta de Moebio
Cuadernos de Economía
El Vigía (Santiago)
Electronic Journal of Biotechnology
Journal of Technology Management
and Innovation
Monografías electrónicas de patología
veterinaria
Política Criminal
Rev. Chilena de Semiótica
Rev. Electrónica de la Sociedad Chilena
de Ciencia de la Computación
Revista Universitaria
Colombia
Acta Biológica Colombiana
Colombia Médica
Cuadernos de Administración
Earth Sciences Research Journal
Livestock Research For Rural
Development
Nómadas
Rev. Ciencias Humanas
334
Rolando Coto, Helena Francke, Saray Córdoba
Rev. Latinoamericana de Ciencias
Sociales, Niñez y Juventud
Revista EIA Ingeniería Antioquía
Revista E-mercatoria
Revista Escuela Colombiana de
Medicina - ECM
Costa Rica
Diálogos
MHSalud
Población y Salud en Mesoamérica
Reflexiones
Rev. Actualidades Investigativas en
Educación
Revista de Derecho Electoral
Cuba
ACIMED
Fitosanidad
Multimed
Revista cubana de investigaciones
biomédicas
Revista Cubana de Obstetricia y
Ginecología
Revista cubana de pediatría
Ecuador
Gaceta Dermatológica Ecuatoriana
(Ecuador)
Universidad-Verdad (Ecuador)
Mexico
Acta Médica Grupo Ángeles
Alegatos
Aleph Zero
Anales del I.Biología, Serie Zoología
Archivos Hispanoamericanos de
Sexología
Biblioteca Universitaria
Buenaval
Computación y Sistemas
Cuadernos de Psicoanálisis
Dugesiana
Educar
e-Gnosis
El Psicólogo de Anahuac
Hitos de Ciencias Económico
Administrativas
InFÁRMAte
Investigación Bibliotecológica
Journal of Applied Research and
Technology
La pintura mural prehispánica en
México. Boletin informativo
Los amantes de Sofía
Mensaje bioquímico
Nueva Antropología
Redes Música
Rev. Ciencia Veterinaria
Revista Biomédica
Revista de Enfermedades Infecciosas en
Pediatría
Revista de la Educación Superior
Revista del Instituto Nacional de
Cancerología
Revista Mexicana de Física
Peru
Biblios
Diagnóstico
Escritura y Pensamiento
Puerto Rico
Ceteris Paribus
El Amauta
Rev. Int. Desastres Naturales,
Infraestructura Civil
Videoenlace Interactivo
Uruguay
Actas de Fisiología
Boletín Cinterfor
Boletín del Inst. de Inv. Pesqueras
Galileo
Venezuela
Acción Pedagógica
Boletín Antropológico
Cayapa
Música en clave
Postgrado
Rev. Ingeniería UC
Revista de la Sociedad MédicoQuirúrgica del Hospital de
Emergencia Pérez de León
June 2009
Printed on demand
by "Nuova Cultura"
www.nuovacultura.it
Book orders:
[email protected]