#FactCheck - Deepfake Video Falsely Claims visuals of a massive rally held in Manipur

In the rich history of humanity, the advent of artificial intelligence (AI) has added a new, delicate aspect. The aspect of promising technological advancement has the potential to either enrich the nest of our society or destroy it entirely. The latest straw in this complex nest is generative AI, a frontier teeming with both potential and perils. It is a realm where the ethereal concepts of cyber peace and resilience are not just theoretical constructs but tangible necessities.

The spectre of generative AI looms large over the digital landscape, casting a long shadow on the sanctity of data privacy and the integrity of political processes. The seeds of this threat were sown in the fertile soil of the Cambridge Analytica scandal of 2018, a watershed moment that unveiled the extent to which personal data could be harvested and utilized to influence electoral outcomes. However despite the indignation, the scandal resulted in meagre alterations to modus operandi of digital platforms.

Fast forward to the present day, and the spectre has only grown more ominous. A recent report by Human Rights Watch has shed light on the continued exploitation of data-driven campaigning in Hungary's re-election of Viktor Orbán. The report paints a chilling picture of political parties leveraging voter databases for targeted social media advertising, with the ruling Fidesz party even resorting to the unethical use of public service data to bolster its voter database.

The Looming Threat of Disinformation

As we stand on the precipice of 2024, a year that will witness over 50 countries holding elections, the advancements in generative AI could exponentially amplify the ability of political campaigns to manipulate electoral outcomes. This is particularly concerning in countries where information disparities are stark, providing fertile ground for the seeds of disinformation to take root and flourish.

The media, the traditional watchdog of democracy, has already begun to sound the alarm about the potential threats posed by deepfakes and manipulative content in the upcoming elections. The limited use of generative AI in disinformation campaigns has raised concerns about the enforcement of policies against generating targeted political materials, such as those designed to sway specific demographic groups towards a particular candidate.

Yet, while the threat of bad actors using AI to generate and disseminate disinformation is real and present, there is another dimension that has largely remained unexplored: the intimate interactions with chatbots. These digital interlocutors, when armed with advanced generative AI, have the potential to manipulate individuals without any intermediaries. The more data they have about a person, the better they can tailor their manipulations.

Root of the Cause 

To fully grasp the potential risks, we must journey back 30 years to the birth of online banner ads. The success of the first-ever banner ad for AT&T, which boasted an astounding 44% click rate, birthed a new era of digital advertising. This was followed by the advent of mobile advertising in the early 2000s. Since then, companies have been engaged in a perpetual quest to harness technology for manipulation, blurring the lines between commercial and political advertising in cyberspace.

Regrettably, the safeguards currently in place are woefully inadequate to prevent the rise of manipulative chatbots. Consider the case of Snapchat's My AI generative chatbot, which ostensibly assists users with trivia questions and gift suggestions. Unbeknownst to most users, their interactions with the chatbot are algorithmically harvested for targeted advertising. While this may not seem harmful in its current form, the profit motive could drive it towards more manipulative purposes.

If companies deploying chatbots like My AI face pressure to increase profitability, they may be tempted to subtly steer conversations to extract more user information, providing more fuel for advertising and higher earnings. This kind of nudging is not clearly illegal in the U.S. or the EU, even after the AI Act comes into effect. The market size of AI in India is projected to touch US$4.11bn in 2023. 

Taking this further, chatbots may be inclined to guide users towards purchasing specific products or even influencing significant life decisions, such as religious conversions or voting choices. The legal boundaries here remain unclear, especially when manipulation is not detectable by the user. 

The Crucial Dos/Dont's 

It is crucial to set rules and safeguards in order to manage the possible threats related to manipulative chatbots in the context of the general election in 2024.

First and foremost, candor and transparency are essential. Chatbots, particularly when employed for political or electoral matters, ought to make it clear to users what they are for and why they are automated. By being transparent, people are guaranteed to be aware that they are interacting with automated processes.

Second, getting user consent is crucial. Before collecting user data for any reason, including advertising or political profiling, users should be asked for their informed consent. Giving consumers easy ways to opt-in and opt-out gives them control over their data.

Furthermore, moral use is essential. It's crucial to create an ethics code for chatbot interactions that forbids manipulation, disseminating false information, and trying to sway users' political opinions. This guarantees that chatbots follow moral guidelines.

In order to preserve transparency and accountability, independent audits need to be carried out. Users might feel more confident knowing that chatbot behavior and data collecting procedures are regularly audited by impartial third parties to ensure compliance with legal and ethical norms.

Important "don'ts" to take into account. Coercion and manipulation ought to be outlawed completely. Chatbots should refrain from using misleading or manipulative approaches to sway users' political opinions or religious convictions.

Another hazard to watch out for is unlawful data collecting. Businesses must obtain consumers' express agreement before collecting personal information, and they must not sell or share this information for political reasons.

At all costs, one should steer clear of fake identities. Impersonating people or political figures is not something chatbots should do because it can result in manipulation and false information.

It is essential to be impartial. Bots shouldn't advocate for or take part in political activities that give preference to one political party over another. In encounters, impartiality and equity are crucial.

Finally, one should refrain from using invasive advertising techniques. Chatbots should ensure that advertising tactics comply with legal norms by refraining from displaying political advertisements or messaging without explicit user agreement.

Present Scenario

As we approach the critical 2024 elections and generative AI tools proliferate faster than regulatory measures can keep pace, companies must take an active role in building user trust, transparency, and accountability. This includes comprehensive disclosure about a chatbot's programmed business goals in conversations, ensuring users are fully aware of the chatbot's intended purposes.

To address the regulatory gap, stronger laws are needed. Both the EU AI Act and analogous laws across jurisdictions should be expanded to address the potential for manipulation in various forms. This effort should be driven by public demand, as the interests of lawmakers have been influenced by intensive Big Tech lobbying campaigns.

At present, India doesn’t have any specific laws pertaining to AI regulation. Ministry of Electronics and Information Technology (MEITY), is the executive body responsible for AI strategies and is constantly working towards a policy framework for AI. The Niti Ayog has presented seven principles for responsible AI which includes equality , inclusivity, safety, privacy, transparency, accountability, dependability and protection of positive human values.

Conclusion 

We are at a pivotal juncture in history. As generative AI gains more power, we must proactively establish effective strategies to protect our privacy, rights and democracy. The public's waning confidence in Big Tech and the lessons learned from the techlash underscore the need for stronger regulations that hold tech companies accountable. Let's ensure that the power of generative AI is harnessed for the betterment of society and not exploited for manipulation.

Reference

McCallum, B. S. (2022, December 23). Meta settles Cambridge Analytica scandal case for $725m. BBC News. https://www.bbc.com/news/technology-64075067

Hungary: Data misused for political campaigns. (2022, December 1). Human Rights Watch. https://www.hrw.org/news/2022/12/01/hungary-data-misused-political-campaigns

Statista. (n.d.). Artificial Intelligence - India | Statista Market forecast. https://www.statista.com/outlook/tmo/artificial-intelligence/india

Introduction
‍

Generative AI, particularly deepfake technology, poses significant risks to security in the financial sector. Deepfake technology can convincingly mimic voices, create lip-sync videos, execute face swaps, and carry out other types of impersonation through tools like DALL-E, Midjourney, Respeecher, Murf, etc, which are now widely accessible and have been misused for fraud. For example, in 2024, cybercriminals in Hong Kong used deepfake technology to impersonate the Chief Financial Officer of a company, defrauding it of $25 million. Surveys, including Regula’s Deepfake Trends 2024 and Sumsub reports, highlight financial services as the most targeted sector for deepfake-induced fraud.

Deepfake Technology and Its Risks to Financial Systems

India’s financial ecosystem, including banks, NBFCs, and fintech companies, is leveraging technology to enhance access to credit for households and MSMEs. The country is a leader in global real-time payments and its digital economy comprises 10% of its GDP.  However, it faces unique cybersecurity challenges. According to the RBI’s 2023-24 Currency and Finance report, banks cite cybersecurity threats, legacy systems, and low customer digital literacy as major hurdles in digital adoption. Deepfake technology intensifies risks like:

• Social Engineering Attacks: Information security breaches through phishing, vishing, etc. become more convincing with deepfake imagery and audio.
• Bypassing Authentication Protocols: Deepfake audio or images may circumvent voice and image-based authentication systems, exposing sensitive data.
• Market Manipulation: Misleading deepfake content making false claims and endorsements can harm investor trust and damage stock market performance.
• Business Email Compromise Scams: Deepfake audio can mimic the voice of a real person with authority in the organization to falsely authorize payments.
• Evolving Deception Techniques: The usage of AI will allow cybercriminals to deploy malware that can adapt in real-time to carry out phishing attacks and inundate targets with increased speed and variations. Legacy security frameworks are not suited to countering automated attacks at such a scale.

Existing Frameworks and Gaps

In 2016, the RBI introduced cybersecurity guidelines for banks, neo-banking, lending, and non-banking financial institutions, focusing on resilience measures like Board-level policies, baseline security standards, data leak prevention, running penetration tests, and mandating Cybersecurity Operations Centres (C-SOCs). It also mandated incident reporting to the RBI for cyber events. Similarly, SEBI’s Cybersecurity and Cyber Resilience Framework (CSCRF) applies to regulated entities (REs) like stock brokers, mutual funds, KYC agencies, etc., requiring policies, risk management frameworks, and third-party assessments of cyber resilience measures. While both frameworks are comprehensive, they require updates addressing emerging threats from generative AI-driven cyber fraud.

Cyberpeace Recommendations 

• AI Cybersecurity to Counter AI Cybercrime: AI-generated attacks can be designed to overwhelm with their speed and scale. Cybercriminals increasingly exploit platforms like LinkedIn, Microsoft Teams, and Messenger, to target people.  More and more organizations of all sizes will have to use AI-based cybersecurity for detection and response since generative AI is becoming increasingly essential in combating hackers and breaches.
• Enhancing Multi-factor Authentication (MFA):  With improving image and voice-generation/manipulation technologies, enhanced authentication measures such as token-based authentication or other hardware-based measures, abnormal behaviour detection, multi-device push notifications, geolocation verifications, etc. can be used to improve prevention strategies.  New targeted technological solutions for content-driven authentication can also be implemented.
• Addressing Third-Party Vulnerabilities: Financial institutions often outsource operations to vendors that may not follow the same cybersecurity protocols, which can introduce vulnerabilities. Ensuring all parties follow standardized protocols can address these gaps.
• Protecting Senior Professionals: Senior-level and high-profile individuals at organizations are at a greater risk of being imitated or impersonated since they hold higher authority over decision-making and have greater access  to sensitive information. Protecting their identity metrics through technological interventions is of utmost importance.
• Advanced Employee Training:  To build organizational resilience, employees must be trained to understand how generative and emerging technologies work. A well-trained workforce can significantly lower the likelihood of successful human-focused human-focused cyberattacks like phishing and impersonation.
• Financial Support to Smaller Institutions: Smaller institutions may not have the resources to invest in robust long-term cybersecurity solutions and upgrades. They require financial and technological support from the government to meet requisite standards.

Conclusion

According to The India Cyber Threat Report 2025 by the Data Security Council of India (DSCI) and Seqrite, deepfake-enabled cyberattacks, especially in the finance and healthcare sectors, are set to increase in 2025. This has the potential to disrupt services, steal sensitive data, and exploit geopolitical tensions, presenting a significant risk to the critical infrastructure of India. 

As the threat landscape changes,  institutions will have to continue to embrace AI and Machine Learning (ML) for threat detection and response. The financial sector must prioritize robust cybersecurity strategies, participate in regulation-framing procedures, adopt AI-based solutions, and enhance workforce training, to safeguard against AI-enabled fraud. Collaborative efforts among policymakers, financial institutions, and technology providers will be essential to strengthen defenses.

‍

Sources 

• https://sumsub.com/newsroom/deepfake-cases-surge-in-countries-holding-2024-elections-sumsub-research-shows/ 
• https://www.globenewswire.com/news-release/2024/10/31/2972565/0/en/Deepfake-Fraud-Costs-the-Financial-Sector-an-Average-of-600-000-for-Each-Company-Regula-s-Survey-Shows.html 
• https://www.sipa.columbia.edu/sites/default/files/2023-05/For%20Publication_BOfA_PollardCartier.pdf 
• https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html 
• https://www.rbi.org.in/Commonman/English/scripts/Notification.aspx?Id=1721 
• https://elplaw.in/leadership/cybersecurity-and-cyber-resilience-framework-for-sebi-regulated-entities/ 
• https://economictimes.indiatimes.com/tech/artificial-intelligence/ai-driven-deepfake-enabled-cyberattacks-to-rise-in-2025-healthcarefinance-sectors-at-risk-report/articleshow/115976846.cms?from=mdr 

‍

Introduction‍

Netizens across the globe have been enjoying the fruits of technological advancements in the digital century. Our personal and professional life has been impacted deeply by the new technologies. The previous year we saw an exponential rise in blockchain integration and the applications of Web 3.0. There is no denying that the Covid-19 pandemic caused a rapid rise in technology and internet penetration all across the globe, bringing the world closer with respect to connectivity and the exchange of ideas and knowledge. Tech advancements have definitely made our lives easier, but the same has also opened the doors to various vulnerabilities and new potential threats. As cyberspace expands, so do the vulnerabilities associated with it, and it is critical we take note of such issues and create safeguards to the extent that such incidents are prevented before they occur. We need to create sustainable and secure cyberspace for future generations.MetaVerse in 2023The metaverse was introduced by Facebook (now Meta) in 2021 as a peak into the future of cyberspace. Since then, tech developers have been working towards arming the metaverse with extraordinary innovations and applications. Netizens came across news like someone bought a house or a plot in the metaverse, someone bought a car in the metaverse, and so on, these news were taken to be the evidence of the netizen’s transition towards the new digital age as we have seen in sci-fi movies. But today this type of news has become history and the metaverse is expanding faster than ever. Let us look at the latest developments and trends in the metaverse-

• Avatar creation - The avatar creation in the metaverse will be a pivotal move as the avatars will represent the user, and essentially it will be the digital, version of the user and will be similar to the user's personal and physical traits to maintain realism in the metaverse.
• Architecture firms - Metaverse has its own set of architects who will be working towards creating your dream home or pro[erty in the metaverse, the heavy code-based services are now being sold just as if they were in the physical space.
• Mining - The metaverse already has companies who are mining gold, silver, petroleum, and other resources for the avatars in the metaverse, for instance, if someone has bought a car in the metaverse, it will still need fuel to run.
• Security firms - These firms are the first line of defenders in the metaverse as they provide tech-based solutions and protocols to secure one’s avatar and belongings in the metaverse.
• Metaverse Police - Interpol, along with its global partner organization has created the metaverse police, who will be working towards creating a safe cyber ecosystem by maintaining compliance with digital laws and ethics.

Advancements beyond metaverse in 2023

‍Technology continues to be a critical force for change in the world. Technology breakthroughs give enterprises more possibilities to lift their productivity and invent offerings. And while it remains difficult to forecast how technology trends will play out, business leaders can plan ahead better by watching the development of new technologies, anticipating how companies could utilize them, and understanding the factors that impact innovation and adoption.

• Applied observability

It advances the practice of pattern recognition. To foresee and identify abnormalities and offer solutions, one must have the capacity to delve deeply into complicated systems and a stream of data. Data fuels this aspect of tech growth in the future.

• Digital Immune System

To ensure that all major systems operate round-the-clock to deliver uninterrupted services, Digital Immune System will combine observability, AI-augmented testing, chaos engineering, site reliability engineering (SRE), and software supply chain security. This will take the efficiency of the systems to a new level.

• Super apps

These represent the upcoming shift in application usage, design, and development, where consumers will utilise a single app to manage most systems in an enterprise ecosystem. Over 50% of the world’s population will utilise super apps on a daily basis to fulfill their daily personal and professional needs.

• AR/VR and BlockChain technology

A combination of better interconnected, safe, and immersive virtual environments where people and businesses may recreate real-life scenarios will be created by combining AR/VR, AI/ML, IoT, and Blockchain, thus creating a new vertical of innovation with keen technologies of Web 3.0.

• AAI

The next level of AI, i.e., Advanced Artificial Intelligence (AI), will revolutionise machine learning, pattern recognition, and computing. It aims to fully automate processes without requiring any manual input, thus eradicating the issues of human error and bad actor influence completely.

• Corporate Metaverse

Aside from its power as a marketing tool, the metaverse promises to provide platforms, tools, and entire virtual worlds where business can be done remotely, efficiently, and intelligently. We can expect to see the metaverse concept merge with the idea of the “digital twin” – virtual simulations of real-world products, processes, or operations that can be used to test and prototype new ideas in the safe environment of the digital domain. From wind farms to Formula 1 cars, designers are recreating physical objects inside virtual worlds where their efficiency can be stress-tested under any conceivable condition without the resource costs that would be incurred by testing them in the physical world.ConclusionIn 2023, we will see more advanced use cases for technology such as motion capture, which will mean that as well as looking and sounding more like us, our avatars will adopt our own unique gestures and body language. We may even start to see further developments in the fields of autonomous avatars – meaning they won't be under our direct control but will be enabled by AI to act as our representatives in the digital world while we ourselves get on with other, completely unrelated tasks. As we go deeper into cyberspace, we need to remember the basic safety practices and inculcate them with respect to cyberspace and work towards creating string policies and legislations to safeguard the digital rights and duties of the netizen to create a wholesome and interdependent cyber ecosystem.

‍