Montenegro’s Judicial Council said on Wednesday that the court system had been slowed down by the massive cyber-attacks which have hit the country since August 22.
The digital infrastructure of a major part of Montenegro’s public administration has been offline since August 22 following an unprecedented series of a ransomware attacks on government servers.
The Judicial Council told BIRN that some trials had to be postponed due to technical problems.
“The reason for the postponement of certain trials lies in the fact that a small number of users are unable to access the Judicial Information System, PRIS, and enter the necessary data due to problems with the links provided by the Ministry of Public Administration.
“We are actively working on finding alternative ways to access the system for users who do not currently have it, until the establishment of a regular operating regime,” the Judicial Council’s press office said.
“So far, no data were compromised in the court system, nor was direct damage to the system detected,” it added.
On September 5, the Higher Court in Podgorica postponed the so-called “coup plot” trial, as it couldn’t provide technical conditions for the trial. It postponed the trial to November 28.
After the second cyber-attack on August 26, certain services were switched off temporarily for security reasons, causing problems in the functioning of the public administration. The websites of the government and the president are still offline.
Courts and the prosecution service are also working only offline, as are the State Property Administration, the Central Register of Business Entities and the fiscal system.
Government servers were hit with ransomware, a type of malware attack in which the attacker locks and encrypts the target’s data and important files and then demands a payment to unlock and decrypt the data. The head of State Cyber Security Service, Dusan Polovic, said on September 5, that some service could be switched to online mode in future days, stressing that a team from the FBI had joined the investigation.
“In the next few days, we expect developments. Citizens should understand the complexity of the attack. The situation we have, many countries have gone through,” Polovic told television Vijesti.
After the second wave of cyber-attacks on August 26, the Agency for National Security, ANB, accused Russian services of organizing the attacks. On September 1, Russia’s Foreign Ministry dismissed the claims as part of a “continuous policy of dismantling relations with Moscow in order to please the United States”.
On August 31, Public Administration Minister Marash Dukaj blamed Cuba Ransomware for the attack. He said this group had created a special virus for this attack, costing about 10 million dollars and which has not been used anywhere so far.
Montenegrin Minister of Public Administration Marash Dukaj said on Monday that organized cyber attacks on government servers have continued, adding that the damage to public data still has to be assessed.
Since August 22, the government has reported two series of cyber-attacks on government servers, claiming they managed to prevent any damage.
“The damage is being repaired and we are assessing its extent. The system will suffer no lasting effects. A huge amount of money was invested in this attack on our system,” Dukaj told a press conference.
Head of State Cyber Security Service Dusan Polovic said the authorities are not able to activate some services online, and a certain number of workstations are compromised.
“The cost of the virus used for the first attacks on the dark web is from 100 thousand to 2.5 million dollars,” said Polovic.
On August 26, the Ministry of Public Administration said some government servers were temporarily taken offline, while the Agency for National Security, ANB, accused Russian services of organizing coordinated cyber-attacks on government servers. The ANB said Montenegro was caught up in a “hybrid war”, claiming that an attack had been prepared for a long time.
The ANB did not respond to BIRN requests about the cyber-attacks’ investigation’s results. The head of the Electric Company, Milutin Djukanovic, on Monday meanwhile said ANB chief Savo Kentera had warned him about potential cyber-attacks on the electricity system, so they switched to a manual operating system.
On August 26, the US embassy in Montenegro warned its citizens that cyber-attacks may include disruptions to public utility, transportation and telecommunication sectors.
After a National Security session on August 26, outgoing Prime Minister Dritan Abazovic described the cyber-attacks as dangerous, but added that citizens’ personal data were safe. He said the authorities don’t have firm evidence yet about the organizers of the attack.
“We do not have clear information about the organizers. Security sector authorities couldn’t confirm that there is an individual, a group, a state behind this, nor could we deny it,” Abazovic said.
The government published a safe protocol for safety in cyberspace, calling on citizens to use licensed operative systems and create backup copies of all important data. The government noted that NATO members have helped Montenegrin authorities to prevent cyber-attack damage.
Reportedly, government servers were hit by ransomware, a type of malware attack in which the attacker locks and encrypts the victim’s data and important files, and then demands a payment to unlock and decrypt the data.
On Monday, Veselin Konatar, a professor from the University of Podgorica, said the government had not provided firm evidence about the cyber-attacks’ organizers.
“There is a real possibility that a cyber-attack on the government’s IT infrastructure could have been organized by both individuals and organized criminal groups… Also, the government surprisingly quickly assessed that there was no permanent damage to the IT infrastructure, nor any compromise of citizens’ data, which requires much more time to confirm,” Konatar told the daily Dan.
On Monday, IT specialist Branko Popovic urged authorities to present the results of the cyber-attacks investigation, warning also that the government doesn’t have the administrative capacities to deal with such attacks. “It’s possible that someone deliberately released a virus into the government servers in order to steal confidential information, correspondence or reports,” he posted on Facebook.
The government has not adopted a new Cyber Security Strategy after the last one became outdated in 2021. In July 2021, the then Minister of Public Administration, Digital Society and Media, Tamara Srzentic, said that the government would improve its administrative capacities in the cyber security sector, and push for international cooperation and staff education.
The Montenegrin Supreme Court said on Thursday that prominent investigative journalist Jovo Martinovic, who was convicted of drug trafficking in a case that has sparked criticism from the EU, should be retried for a second time.
“The decision has been made and at the beginning of next week, [the case] will be returned to the lower court,” the Supreme Court told daily newspaper Vijesti.
In the second-instance verdict in the case in October 2020, Montenegrin Higher Court found Martinovic guilty of mediation in drug trafficking and sentenced to a year in prison.
Martinovic said he will continue to try to prove his innocence.
“Unfortunately, in Montenegro, it is not up to the prosecutor to prove guilt, but up to journalists to prove their innocence. This retrial could be good for media freedom and I will continue to fight,” Martinovic told BIRN.
He was arrested in October 2015 alongside 17 others from Montenegro in a joint police operation conducted with Croatian police. He spent almost a year-and-a-half in custody before being released in January 2017 ahead of the trial.
He was convicted in the first-instance ruling in January 2019 of drug trafficking and membership of a criminal organisation but the Appeals Court overturned the verdict and ordered the first retrial.
The journalist always insisted he had made contacts with alleged drug traffickers only as part of his legitimate reporting work.
Martinovic made contacts with two of the 17 suspects arrested in 2015: Dusko Martinovic – no relation to the journalist – and Namik Selmanovic.
Dusko Martinovic, the main suspect in the case, was also a convicted member of a gang of jewel thieves known as the Pink Panthers. Operating in the US, Europe, the Middle East and Asia, they are believed to have stolen hundreds of millions of euros’ worth of jewellery. Journalist Martinovic worked with Dusko Martinovic on a series of TV shows about the robbers produced by Vice media group.
He worked alongside Selmanovic when a French production company, CAPA Presse, hired them to contribute to research on a documentary about weapons smuggling.
Dusko Martinovic was sentenced to six years and three months in prison in January 2019. Selmanovic has turned state’s evidence.
The European Commission’s report last year about Montenegro’s progress towards membership warned that the conviction of the journalist raises concerns about reporters’ ability to perform their duties professionally and without fear of legal repercussions in the country.
“Cetinje is our light of freedom … we are coming !!!” reads one of the tweets published on the eve of the divisive enthronement of a new Serbian Orthodox Church Metropolitan in Cetinje, the former royal capital of Montenegro.
The tweet was one of thousands published between September 4 and 5 as violent clashes between police and opponents of the event rocked the country, and led to accusations that the opposition was fomenting a coup against the new government elected last year.
BIRN noted more than 4,150 tweets published over the weekend containing words “komite”, “ustolicenje” [“enthronment”] or “Cetinje”.
“Komitas” is a nickname for Montenegrins who are especially fierce in their defence of Montenegro’s statehood, the Montenegrin language and Church, their name recalling the resistance to Montenegro’s union with the Kingdom of Serbia in the wake of World War I.
Montenegro remained united with Serbia until it regained independence in 2006.
During the weekend, the word “ustolicenje” was mentioned in tweets 1,374 times, “Cetinje” 3,042 times and “komite” 499 times. BIRN used a browser add-on Tweet archiver to extract tweets mentioning these key words.
The dramatic rise in use of the term “komite” is illustrated by the fact that in the first seven months of this year “komite” was used more than 1,100 times, slightly more than twice the number of mentions over the past weekend alone.
Whether the online dispute contributed to the weekend’s violent events in Cetinje is hard to tell. Nevertheless, the violence that erupted in Cetinje as opposition parties and self-declared patriotic groups battled police while trying to stop the enthronement of the new head of the Serbian Orthodox Church was clearly echoed on social media.
The phrase “Cetinje will not fall !!!!!” was tweeted 14 times, while “Next year in Cetinje” [referring to “Dogodine u Prizrenu” often used to illustrate Serbian nationalists’ wish to regain control of Kosovo] was tweeted four times.
The most retweeted and liked tweets were one in favour of the new Metropolitan’s enthronement, and another defending the protesters in Cetinje.
The first was posted by Mustafa Jusufspahic, Muslim mufti of Belgrade, in which he wrote: “I congratulate the [Serbian Orthodox] Church, my clergy, and followers of the SOC, on the enthronement of the worthy Metropolitan of Montenegro and the Littoral, Joanikije.”
The other was posted by Ivan Vukovic, mayor of the capital, Podgorica, blaming Prime Minister Krivokapic for what he believed was unnecessary police violence: “I appealed to, begged @profKrivokapic … In vain. He proved unworthy of his office and entered the history of Montenegro as the one who, fabricating Molotov cocktails, gave the order to run over Cetinje and, as never before, humiliate our state. You rejoice today, but you will be ashamed while you yet llive.”
Many tweeters shared videos documenting the clashes between protesters and police.
Apelovao sam, molio @profKrivokapic … uzalud. Pokazao se nedostojnim funkcije i upisao se u istoriju Crne Gore kao onaj koji je, izmišljajući molotovljeve koktele, dao nalog da se pregazi Cetinje i, kao nikad, ponizi naša država.
A war of words on social media in Montenegro, mainly between pro- and anti-Serbian factions, took off in 2019 and intensified after elections last year ousted the long ruling pro-independence Democratic Party of Socialists, DPS. It peaked this weekend, with more than 4,150 tweets in just 48 hours.
In August, BIRN published an investigation examining the rise in online hate speech in Montenegro that turned into offline violence on Sunday.
Searching Twitter for six specific terms or hashtags used, BIRN extracted more than 7,000 local-language tweets between November 1, 2019 – when the row over a disputed religious law took off – and July 19 this year.
The six terms and hashtags used were “osvjezilo” [refresh], “#nedamosvetinje” [roughly, “We won’t give up our shrines”], “komite”, “#nikadvise” [“never again”], “FCJK” [“Faculty for Montenegrin Language and Literature in Cetinje] and “#crnogorskoprolece” [“Montenegrin Spring”].
The results showed a continued significant rise in their use over the period monitored.
“Komite”, for example, was used only 20 times in 2019, 830 times in 2020 and 1,104 times in the first seven months of this year. “Osvjezilo” was mentioned only four times in 2019, 161 times in 2020 and 1,091 times until July 19 this year.
The enthronement of the new Metropolitan of Montenegro has angered members of self-proclaimed patriotic organizations, called “komitas”, and members of the opposition parties, who claim that holding an event with strong Serbian connotations in the old royal capital is an insult to Montenegro’s “centuries-long struggle for sovereignty”.
In April, self-proclaimed patriotic organisations were already organising protests against the new government, in which pro-Serbian parties play an important role, accusing it of undermining Montenegro’s statehood.
The outgoing Bishop of Budimlja-Niksic was elected the new Metropolitan of Montenegro at the Bishops’ Council of the Serbian Church in Belgrade on May 29, to succeed the late Metropolitan Amfilohije, who died of coronavirus infection last year.
Montenegro, which declared independence in 2006, is a multi-ethnic society, deeply split between those who consider themselves Montenegrins, those who identify as Serbs and various other smaller groups.
Society is also split on religious lines. According to the last census from 2011, 72 per cent of Montenegrin citizens identify themselves as Orthodox Christians, and about 70 per cent of this number follow the Serbian Orthodox Church and 30 per cent identify with the Montenegrin Orthodox Church, which is not canonically recognised by other Orthodox Churches.
A member of Montenegro’s Council of the Agency for Personal Data Protection, Muhamed Gjokaj, on Wednesday warned that new COVID-19 measures could put citizens’ personal data at risk.
He said he feared unauthorized persons could get insight into citizens’ personal data, and called on the Health Ministry to be more precise about its new health measures.
“The Health Ministry should explain on the basis of which specific legal norms it has prescribed that waiters have the right to process the personal data of citizens who enter a café or restaurant.
“If there is no adequate legal basis, citizens can sue all those entities that ask to inspect their personal data, which also relates to health information,” Gjokaj told the daily Pobjeda.
On July 30, the Health Ministry announced that patrons of nightclubs, discotheque and indoor restaurants must show their ID and National COVID-19 certificate before entering.
The national COVID-19 certificate is a document issued by Health Ministry, which proves that a person has been vaccinated, or has had a recent negative PCR test, or has recovered from COVID-19. According to the Health Ministry, the certificate must be showed to the waiter or club staff.
Montenegro’s Personal Data Protection Law specifies that personal data related to health conditions can be inspected only by medical personnel, however. It prohibits inspection of personal data by unauthorized personnel.
On July 30, the head of the Digital Health Directorate, Aleksandar Sekulic, said no violation of citizens’ personal data was taking place under the measures, as only the name and date of birth of the person were on the COVID-19 certificate.
“We do not provide medical conditions through the certificates but only the data citizens want to provide. They voluntarily agreed to provide a certain amount of data,” Sekulic told a press conference.
On August 3, a lawyer, Andrijana Razic claimed the Health Ministry had violated the law by the new health measures, accusing it of forcing citizens to be vaccinated. She said that non-vaccinated citizens must not be discriminated against in any way.
“It’s completely clear that employees in a restaurant or nightclub have absolutely no right to identify citizens, or ask them for health information that is secret by law. The government should seriously consider the possible consequences of pursuing such a discriminatory and dangerous health policy, based on a drastic violation of basic human rights,” Razic told the daily newspaper Dan.
According to the Institute for Public Health, there are 1,667 registered COVID-19 active cases in the country. The capital Podgorica and the coastal town of Budva have the largest numbers. On Wednesday, the Health Ministry said that 34.5 per cent of the adult population had been vaccinated against COVID-19.
The head of the prominent Montenegrin watchdog MANS, Vanja Calovic Markovic, on Wednesday alleged that most political parties concealed part of their election funding sources and the actual costs of their campaigns for last year’s parliamentary elections.
At the promotion of MANS’ parliamentary election report, Calovic Markovic also said one of the parties in the now ruling For the Future of Montenegro coalition used an offshore company to make camaign videos.
“They hired an offshore company, Limanaki Studios LTD, from Cyprus, to produce videos. That company has not submitted financial reports since its establishment, and its owners are hidden,” Calovic Markovic said, not naming the actual party.
“The contract specifies the given company to do advertising videos for 50,000 euros, without the number of commercials or the deadline by when it must be completed, only the payment deadline,” she added.
In parliamentary elections held on August 30 last year, three opposition blocs won a slender majority of 41 of the 81 seats in parliament, ousting the long-ruling Democratic Party of Socialists, DPS.
New Prime Minister Zdravko Krivokaic led the largest bloc in the now ruling majority, the pro-Serbian For the Future of Montenegro, with the Democratic Front, DF, as its strongest member.
DF officials and the Prime Minister have not commented on the MANS report.
Calovic Markovic said that all the competing electoral lists spent a total of about 245,000 euros on advertising on social networks, almost ten times more than in the 2016 parliamentary elections.
“The administrators on social networks that financed the advertising of certain political structures were exclusively from Serbia,” Calovic Markovic noted.
In December 2020, in its report on the election, the OSCE/ODIHR mission said that Montenegro had not managed to ensure transparency, accountability and integrity of campaign finances, despite changes to electoral laws.
“There is general public mistrust in the campaign finance regulatory system, as currently implemented, and despite some improvements, the legal framework does not establish effective safeguards against corruption or circumvention of campaign finance rules,” the report said.
In its 2020 progress report on the country, the European Commission warned that the electoral legal framework remained largely unchanged since the parliamentary elections in 2016.
“While it provides basic regulations for the conduct of democratic elections, gaps and ambiguities allow for circumvention, particularly in campaign finance,” it observed.
Suspicions about secret camaign funds grew in January 2019, when a video clip from 2016 surfaced in which Dusko Knezevic, chairman of the Montenegro-based Atlas Group, appears to hand the then-mayor of Podgorica, Slavoljub Stijepovic, an envelope containing what Knezevic later said was $200,000 to fund election campaigns.
Knezevic, now believed to be in London, told the media he had been giving the DPS, led by President Milo Djukanovic, such sums for about 25 years, during which time the DPS had never been out of power.
In February 2019, the DPS was fined 20,000 euros and ordered to pay 47,500 euros to the state budget, while in February this year the Higher Court in Podgorica suspended criminal proceedings on Stijepovic.
Montenegro’s Interior Ministry on Wednesday called for stricter penalties for attacks on journalists, promising to consider granting them the same status as state officials. Interior Ministry Secretary Zoran Miljanic said authorities would meanwhile investigate the motives behind the recent attack on the weekly Monitor’s editor-in-chief, Esad Kocan.
“The motive for the attack will be determined, but penalties for attacking journalists must also be stricter. The ministry will consider all possibilities about whether journalists should get the status of officials,” Miljanic told a press conference after a session of the Commission for Monitoring Violence against Media.
Kocan was attacked on March 28 in front of his house when Dragutin Sukovic, from Podgorica, first verbally insulted him and then tried to use force. Sukovic has been detained on suspicion of endangering security, while police reported that he has been arrested several times since 2010 for attempted murder, drug dealing, domestic violence and assault on police officers.
Commission head Mihailo Jovovic urged the ministry to resolve the motive of this attack, “whether someone attacked Kocan as a journalist, someone sent him [the attacker] there, or it was an attack by an incurable man. If it is revealed that someone sent him, it would be the first time that the preparator of an attack on a journalist was discovered,” he told the press conference.
On March 29, civic activists, media organisations and political parties called again on the authorities to protect the safety of journalists and saying they should be given the status of officials. The Ministry of Public Administration, Digital Society and Media announced a consultation on this idea.
“It [official status] should be granted … to introduce a stricter sanctions policy, which will have a deterrent effect,” the Southeast European Media Association said in a press release.
Under the criminal code, endangering someone’s security incurs a potential fine or a one-year prison sentence. But endangering the security of officials carries a prison sentence of up to three years.
On March 20, television Vijesti journalist Sead Sadikovic was threatened and then assaulted by a group of five or six people carrying Montenegrin flags during a so-called patriotic rally in the town of Bijelo Polje.
Police detained Nermin Omerovic and Edin Dizdarevic for the attack, while authorities condemned it, stating that “violence is not a sign of recognition of a European and cultural Montenegro”.
In its 2020 progress report, the European Commission warned that progress in addressing violence against journalists and media in the country had been limited, adding that the authorities should investigate attacks against journalists as a priority.
“Authorities are expected to demonstrate zero tolerance for threats or attacks against the media, and should refrain from making statements that are not conducive to freedom of expression,” the report said.
On March 30, the US State Department’s latest human rights report warned that unsolved attacks against journalists remained a significant problem in Montenegro. It said more than two-thirds of the 85 attacks recorded on journalists since 2004 remained unsolved or did not result in sentences.
“Harassment of journalists, including use of physical force, was further reported in the course of 2020. Observers also noted that most of the attacks targeted independent or pro-opposition journalists and media professionals,” the report said.
Partners Serbia, a Belgrade-based NGO that works on initiatives to combat corruption and develop democracy and the rule of the law in the Balkan country, had been on Twitter for more than nine years when, in November 2020, the social media giant suspended its account.
Twitter gave no notice or explanation of the suspension, but Ana Toskic Cvetinovic, the executive director of Partners Serbia, had a hunch – that it was the result of a “coordinated attack”, probably other Twitter users submitting complaints about how the NGO was using its account.
“We tried for days to get at least some information from Twitter, like what could be the cause and how to solve the problem, but we haven’t received any answer,” Toskic Cvetinovic told BIRN. “After a month of silence, we saw that a new account was the only option.”
Twitter lifted the suspension in January, again without explanation. But Partners Serbia is far from alone among NGOs, media organisations and public figures in the Balkans who have had their social media accounts suspended without proper explanation or sometimes any explanation at all, according to BIRN monitoring of digital rights and freedom violations in the region.
Experts say the lack of transparency is a significant problem for those using social media as a vital channel of communication, not least because they are left in the dark as to what can be done to prevent such suspensions in the future.
But while organisations like Partners Serbia can face arbitrary suspension, half of the posts on Facebook and Twitter that are reported as hate speech, threatening violence or harassment in Bosnian, Serbian, Montenegrin or Macedonian remain online, according to the results of a BIRN survey, despite confirmation from the companies that the posts violated rules.
The investigation shows that the tools used by social media giants to protect their community guidelines are failing: posts and accounts that violate the rules often remain available even when breaches are acknowledged, while others that remain within those rules can be suspended without any clear reason.
Among BIRN’s findings are the following:
Almost half of reports in Bosnian, Serbian, Montenegrin or Macedonian language to Facebook and Twitter are about hate speech
One in two posts reported as hate speech, threatening violence or harassment in Bosnian, Serbian, Montenegrin or Macedonian language, remains online. When it comes to reports of threatening violence, the content was removed in 60 per cent of cases, and 50 per cent in cases of targeted harassment.
Facebook and Twitter are using a hybrid model, a combination of artificial intelligence and human assessment in reviewing such reports, but declined to reveal how many of them are actually reviewed by a person proficient in Bosnian, Serbian, Montenegrin or Macedonian
Both social networks adopt a “proactive approach”, which means they remove content or suspend accounts even without a report of suspicious conduct, but the criteria employed is unclear and transparency lacking.
The survey showed that people were more ready to report content targeting them or minority groups.
Experts say the biggest problem could be the lack of transparency in how social media companies assess complaints.
The assessment itself is done in the first instance by an algorithm and, if necessary, a human gets involved later. But BIRN’s research shows that things get messy when it comes to the languages of the Balkans, precisely because of the specificity of language and context.
Distinguishing harsh criticism from defamation or radical political opinions from expressions of hatred and racism or incitement to violence require contextual and nuanced analysis.
Half of the posts containing hate speech remain online
Graphic: BIRN/Igor Vujcic
Facebook and Twitter are among the most popular social networks in the Balkans. The scope of their popularity is demonstrated in a 2020 report by DataReportal, an online platform that analyses how the world uses the Internet.
In January, there were around 3.7 million social media users in Serbia, 1.1 million in North Macedonia, 390,000 in Montenegro and 1.7 million in Bosnia and Herzegovina.
In each of the countries, Facebook is the most popular, with an estimated three million users in Serbia, 970,000 in North Macedonia, 300,000 in Montenegro and 1.4 million in Bosnia and Herzegovina.
Such numbers make Balkan countries attractive for advertising but also for the spread of political messages, opening the door to violations.
The debate over the benefits and the dangers of social media for 21st century society is well known.
In terms of violent content, besides the use of Artificial Intelligence, or AI, social media giants are trying to give users the means to react as well, chiefly by reporting violations to network administrators.
There are three kinds of filters – manual filtering by humans; automated filtering by algorithmic tools and hybrid filtering, performed by a combination of humans and automated tools.
In cases of uncertainty, posts or accounts are submitted to human review before decisions are taken, or after in the event a user complaints about automated removal.
“Today, we primarily rely on AI for the detection of violating content on Facebook and Instagram, and in some cases to take action on the content automatically as well,” a Facebook spokesperson told BIRN. “We utilize content reviewers for reviewing and labelling specific content, particularly when technology is less effective at making sense of context, intent or motivation.”
Twitter told BIRN that it is increasing the use of machine learning and automation to enforce the rules.
“Today, by using technology, more than 50 per cent of abusive content that’s enforced on our service is surfaced proactively for human review instead of relying on reports from people using Twitter,” said a company spokesperson.
“We have strong and dedicated teams of specialists who provide 24/7 global coverage in multiple different languages, and we are building more capacity to address increasingly complex issues.”
In order to check how effective those mechanisms are when it comes to content in Balkan languages, BIRN conducted a survey focusing on Facebook and Twitter reports and divided into three categories: violent threats (direct or indirect), harassment and hateful conduct.
The survey asked for the language of the disputedcontent, who was the target and who was the author, and whether or not the report was successful.
Over 48 per cent of respondents reported hate speech, some 20 per cent reported targeted harassment and some 17 per cent reported threatening violence.
The survey showed that people were more ready to report content targeting them or minority groups.
According to the survey, 43 per cent of content reported as hate speech remained online, while 57 per cent was removed. When it comes to reports of threatening violence, content was removed in 60 per cent of cases.
Roughly half of reports of targeted harassment resulted in removal.
Chloe Berthelemy, a policy advisor at European Digital Rights, EDRi, which works to promote digital rights, says the real-life consequences of neglect can be disastrous.
“For example, in cases of image-based sexual abuse [often wrongly called “revenge porn”], the majority of victims are women and they suffer from social exclusion as a result of these attacks,” Berthelemy said in a written response to BIRN. “For example, they can be discriminated against on the job market because recruiters search their online reputation.”
Content removal – censorship or corrective?
Graphic: BIRN/Igor Vujcic.
According to the responses to BIRN’s questionnaire, some 57 per cent of those who reported hate speech said they were notified that the reported post/account violated the rules.
On the other hand, some 28 per cent said they had received notification that the content they reported did not violate the rules, while 14 per cent received only confirmation that their report was filed.
In terms of reports of targeted harassment, half of people said they received confirmation that the content violated the rules; 16 per cent were told the content did not violate rules. A third of those who reported targeted harassment only received confirmation their report was received.
As for threatening violence, 40 per cent of people received confirmation that the reported post/account violated the rules while 60 per cent received only confirmation their complaint had been received.
One of the respondents told BIRN they had reported at least seven accounts for spreading hatred and violent content.
“I do not engage actively on such reports nor do I keep looking and searching them. However, when I do come across one of these hateful, genocide deniers and genocide supporters, it feels the right thing to do, to stop such content from going further,” the respondent said, speaking on condition of anonymity. “Maybe one of all the reported individuals stops and asks themselves what led to this and simply opens up discussions, with themselves or their circles.”
Although for those seven acounts Twitter confirmed they violate some of the rules, six of them are still available online.
Another issue that emerged is unclear criteria while reporting violations. Basic knowledge of English is also required.
Sanjana Hattotuwa, special advisor at ICT4Peace Foundation agreed that the in-app or web-based reporting process is confusing.
“Moreover, it is often in English even though the rest of the UI/UX [User Interface/User Experience] could be in the local language. Furthermore, the laborious selection of categories is, for a victim, not easy – especially under duress.”
Facebook told BIRN that the vast majority of reports are reviewed within 24 hours and that the company uses community reporting, human review and automation.
It refused, however, to give any specifics on those it employs to review content or reports in Balkan languages, saying “it isn’t accurate to only give the number of content reviewers”.
BIRN methodology
BIRN conducted its questionnaire via the network’s tool for engaging citizens in reporting, developed in cooperation with the British Council.
The anonymous questionnaire had the aim of collecting information on what type of violations people reported, who was the target and how successful the report was. The questions were available in English, Macedonian, Albanian and Bosnian/Serbian/Montenegrin. BIRN focused on Facebook and Twitter given their popularity in the Balkans and the sensitivity of shared content, which is mostly textual and harder to assess compared to videos and photos.
“That alone doesn’t reflect the number of people working on a content review for a particular country at any given time,” the spokesperson said.
Social networks often remove content themselves, in what they call a ‘proactive approach’.
According to data provided by Facebook, in the last quarter of 2017 their proactive detection rate was 23.6 per cent.
“This means that of the hate speech we removed, 23.6 per cent of it was found before a user reported it to us,” the spokesperson said. “The remaining majority of it was removed after a user reported it. Today we proactively detect about 95 per cent of hate speech content we remove.”
“Whether content is proactively detected or reported by users, we often use AI to take action on the straightforward cases and prioritise the more nuanced cases, where context needs to be considered, for our reviewers.”
There is no available data, however, when it comes to content in a specific language or country.
Facebook publishes a Community Standards Enforcement Report on a quarterly basis, but, according to the spokesperson, the company does not “disclose data regarding content moderation in specific countries.”
Whatever the tools, the results are sometimes highly questionable.
In May 2018, Facebook blocked for 24 hours the profile of Bosnian journalist Dragan Bursac after he posted a photo of a detention camp for Bosniaks in Serbia during the collapse of federal Yugoslavia in the 1990s.
Facebook determined that Bursac’s post had violated “community standards,” local media reported.
Bojan Kordalov, Skopje-based public relations and new media specialist, said that, “when evaluating efficiency in this area, it is important to emphasise that the traffic in the Internet space is very dense and is increasing every second, which unequivocally makes it a field where everyone needs to contribute”.
“This means that social media managements are undeniably responsible for meeting the standards and compliance with regulations within their platforms, but this does not absolve legislators, governments and institutions of responsibility in adapting to the needs of the new digital age, nor does it give anyone the right to redefine and narrow down the notion and the benefits that democracy brings.”
Lack of language sensibility
Illustration. Photo: Unsplash/The Average Tech Guy
SHARE Foundation, a Belgrade-based NGO working on digital rights, said the question was crucial given the huge volume of content flowing through the likes of Facebook and Twitter in all languages.
“When it comes to relatively small language groups in absolute numbers of users, such as languages in the former Yugoslavia or even in the Balkans, there is simply no incentive or sufficient pressure from the public and political leaders to invest in human moderation,” SHARE told BIRN.
Berthelemy of EDRi said the Balkans were not a stand alone example, and that the content moderation practices and policies of Facebook and Twitter are “doomed to fail.”
“Many of these corporations operate on a massive scale, some of them serving up to a quarter of the world’s population with a single service,” Berthelemy told BIRN. “It is impossible for such monolithic architecture, and speech regulation process and policy to accommodate and satisfy the specific cultural and social needs of individuals and groups.”
The European Parliament has also stressed the importance of a combined assessment.
“The expressions of hatred can be conveyed in many ways, and the same words typically used to convey such expressions can also be used for different purposes,” according to a 2020 study – ‘The impact of algorithms for online content filtering or moderation’ – commissioned by the Parliament’s Policy Department for Citizens’ Rights and Constitutional Affairs.
“For instance, such words can be used for condemning violence, injustice or discrimination against the targeted groups, or just for describing their social circumstances. Thus, to identify hateful content in textual messages, an attempt must be made at grasping the meaning of such messages, using the resources provided by natural language processing.”
Hattotuwa said that, in general, “non-English language markets with non-Romanic (i.e. not English letter based) scripts are that much harder to design AI/ML solutions around”.
“And in many cases, these markets are out of sight and out of mind, unless the violence, abuse or platform harms are so significant they hit the New York Times front-page,” Hattotuwa told BIRN.
“Humans are necessary for evaluations, but as you know, there are serious emotional / PTSD issues related to the oversight of violent content, that companies like Facebook have been sued for (and lost, having to pay damages).”
Failing in non-English
Illustration. Photo: Unsplash/Ann Ann
Dragan Vujanovic of the Sarajevo-based NGO Vasa prava [Your Rights] criticised what he said was a “certain level of tolerance with regards to violations which support certain social narratives.”
“This is particularly evident in the inconsistent behavior of social media moderators where accounts with fairly innocuous comments are banned or suspended while other accounts, with overt abuse and clear negative social impact, are tolerated.”
For Chloe Berthelemy, trying to apply a uniform set of rules on the very diverse range of norms, values and opinions on all available topics that exist in the world is “meant to fail.”
“For instance, where nudity is considered to be sensitive in the United States, other cultures take a more liberal approach,” she said.
The example of Myanmar, when Facebook effectively blocked an entire language by refusing all messages written in Jinghpaw, a language spoken by Myanmar’s ethnic Kachin and written with a Roman alphabet, shows the scale of the issue.
“The platform performs very poorly at detecting hate speech in non-English languages,” Berthelemy told BIRN.
The techniques used to filter content differ depending on the media analysed, according to the 2020 study for the European Parliament.
“A filter can work at different levels of complexity, spanning from simply comparing contents against a blacklist, to more sophisticated techniques employing complex AI techniques,” it said.
“In machine learning approaches, the system, rather than being provided with a logical definition of the criteria to be used to find and classify content (e.g., to determine what counts as hate speech, defamation, etc.) is provided with a vast set of data, from which it must learn on its own the criteria for making such a classification.”
Users of both Twitter and Facebook can appeal in the event their accounts are suspended or blocked.
“Unfortunately, the process lacks transparency, as the number of filed appeals is not mentioned in the transparency report, nor is the number of processed or reinstated accounts or tweets,” the study noted.
Between January and October 2020, Facebook restored some 50,000 items of content without an appeal and 613,000 after appeal.
Machine learning
As cited in the 2020 study commissioned by the European Parliament, Facebook has developed a machine learning approach called Whole Post Integrity Embeddings, WPIE, to deal with content violating Facebook guidelines.
The system addresses multimedia content by providing a holistic analysis of a post’s visual and textual content and related comments, across all dimensions of inappropriateness (violence, hate, nudity, drugs, etc.). The company claims that automated tools have improved the implementation of Facebook content guidelines. For instance, about 4.4 million items of drug sale content were removed in just the third quarter of 2019, 97.6 per cent of which were detected proactively.
When it comes to the ways in which social networks deal with suspicious content, Hattotuwa said that “context is key”.
While acknowledging advancements in the past two to three years, Hattotuwa said that, “No AI and ML [Machine Learning] I am aware of even in English language contexts can accurately identify the meaning behind an image.”
“With regards to content inciting hate, hurt and harm,” he said, “it is even more of a challenge.”
According to the Twitter Transparency report, in the first six months of 2020, 12.4 million accounts were reported to the company, just over six million of which were reported for hateful conduct and some 5.1 million for “abuse/harassment”.
In the same period, Twitter suspended 925,744 accounts, of which 127,954 were flagged for hateful conduct and 72,139 for abuse/harassment. The company removed such content in a little over 1.9 million cases: 955,212 in the hateful conduct category and 609,253 in the abuse/harassment category.
Toskic Cvetinovic said the rules needed to be clearer and better communicated to users by “living people.”
“Often, the content removal doesn’t have a corrective function, but amounts to censorship,” she said.
Berthelemy said that, “because the dominant social media platforms reproduce the social systems of oppression, they are also often unsafe for many groups at the margins.”
“They are unable to understand the discriminatory and violent online behaviours, including certain forms of harassment and violent threats and therefore, cannot address the needs of victims,” Berthelemy told BIRN.
“Furthermore,” she said, “those social media networks are also advertisement companies. They rely on inflammatory content to generate profiling data and thus advertisement profits. There will be no effective, systematic response without addressing the business models of accumulating and trading personal data.”
We’re looking for people who are willing to share their experience with us to help in a story we’re currently working on. Scroll down for information on how to part take.
The key things we want to know:
What type of violations have you reported?
In what language was the content?
How was the report processed?
What do we consider to be violations of social media community guidelines:
Violent threats (direct or indirect)
Harassment, which entails inciting or engaging in the targeted abuse or harassment of others
Hateful conduct, which entails promoting violence against or directly attacking or threatening other people on the basis of race, ethnicity, national origin, sexual orientation, gender, gender identity, religious affiliation, age, disability or disease.
Things to note:
We are looking for social media users that reported content in the Bosnian, Serbian, Montenegrin, Albanian, and Macedonian languages. We want to hear as many different experiences from all around Southeast Europe.
Your stories will be used to help us with an ongoing investigation.
How to take part?
To submit your experience, all you need to do is fill out this form.
Experts told an online debate hosted by the Balkan Investigative Reporting Network on Tuesday that the current regulation systems for online media in the Western Balkans are not good enough, but efforts to curb the publication of hate speech and defamatory comments must not tip over into censorship.
Media and legal experts from Albania, Bosnia and Herzegovina, Montenegro, North Macedonia and Serbia who spoke at the debate entitled ‘Case Law and Online Media Regulation in the Balkans’ also said that the application of existing legislation is inadequate.
Authorities often rely on legislation that was developed for traditional media which has not been adapted accordingly, or on self-regulation which is not mandatory.
Lazar Sandev, an attorney at law from North Macedonia argued that “those who create public opinion regarding matters of public interest do not uphold any standards, they do not have any legal knowledge”.
Jelena Kleut, associate professor at the University of Novi Sad’s Faculty of Philosophy, said that in Serbia there is lack of willingness to apply standards in online media, and noted a difference between rich and poor media outlets as well as responsible and not responsible ones.
“The wealthy, irresponsible media – they have legal knowledge but they don’t care. They would rather see the complaints in court, pay a certain amount of fines and continue along, they don’t care. On the other end of the spectrum, we have responsible but poor media,” Kleut said.
The media experts also debated the controversial issue of reader comment sections on websites, which some sites around the world have removed in recent years because of a proliferation of hate speech, defamation and insulting language.
According to Montenegro’s Media Law, which came in force in August this year, the founder of an online publication is obliged to remove a comment “that is obviously illegal content” without delay, and no later than 60 minutes from learning or receiving a report that a comment is illegal.
Milan Radovic, programme director of the Civil Alliance NGO and a member of the Montenegrin Public Broadcaster’s governing council, argued that this “it is clear that in such a short period of time, if it is applied, will damage those affected, but also damages for freedom of expression”.
Edina Harbinja, a senior lecturer at Britain’s Aston University, warned that there is a conflict between regulatory attempts and media freedom, and that “this is when we need to be careful in how we regulate, not to result in censorship”.
This was the second debate in a series of discussions on online media regulation with various stakeholders, organised as a part of the regional Media for All project, which aim to support independent media outlets in the Western Balkans to become more audience-oriented and financially sustainable.
The project is funded by the UK government and delivered by a consortium led by the British Council in partnership with BIRN, the Thomson Foundation and the International NGO Training and Research Centre, INTRAC.
BIRD Community
Are you a professional journalist or a media worker looking for an easily searchable and comprehensive database and interested in safely (re)connecting with more than thousands of colleagues from Southeastern and Central Europe?
We created BIRD Community, a place where you can have it all!
