The emerging threat of deepfake technology
Deepfake technology has mainly targeted individuals – particularly women – but there is now a growing expectation that the increased availability of technologies will, inevitably, be used by organised crime against businesses.
By Reuters Plus
At least since Watergate, we have known the power of short media clips to change people’s minds, and the world. Ever since, fiction writers and think tanks have toyed with the idea of how doctoring media could be a threat to individuals, organisations and society.
The rapid advancement of such technologies in the past year or two means we now live in a world where technologies to create highly convincing manipulated media – images, audio and video – are increasingly available to anyone, anywhere, surprisingly cheaply.
A synthesised video of Facebook’s Mark Zuckerberg’s apparently saying sinister things about Facebook’s transparency policy, is the most high-profile example in the corporate world of deepfakery.
According to Giorgio Patrini, the founder CEO of Sensity, a start-up tech company based in the Netherlands, over the past 12 months there were more than 60,000 synthetic videos (typically referred to as ‘deepfakes’) posted online, and the vast majority were malicious, mostly targeting women.
The company specialises in trawling the deep and dark web to identify and remedy such material, and estimates this figure is doubling every six months. If you add still images, the figure is much higher.
During 2020, Sensity uncovered more than 100,000 images on Telegram, a messaging platform similar to WhatsApp, where a Russia-centric bot was maliciously manipulating images of women. “This was a dramatic discovery, and very new, it was not around last year,” says Patrini. “The scale is huge.”
Part of the genesis for this, according to Professor Hany Farid of the University of California, is the unexpectedly rapid criminal adoption of research tools developed by the academic community, exploring the intersection of machine learning technologies and video. These techniques have reduced the cost and complexity of generating convincing manipulated video, and criminal groups have adapted these new tools to nefarious ends.
“Until recently, manipulating video involved computer-generated imagery, a huge budget, Hollywood studios, thousands of people and millions of dollars,” says Farid. “With CGI, you are effectively creating a virtual world, and it’s really hard to do. By contrast, synthesised material uses machines that learn how to create new material by analysing large amounts of existing media.”
In this way, you can synthesise the voices of real people and create video of fake people, as well as ‘face-swap’, ‘puppet master’ and ‘lip synch’ deepfakes. “We’ve always manipulated videos and images and it was difficult, it was in the hands of the few and now it’s in the hands of the many,” says Farid. “That should worry us.”
The fact that such technologies and services are now available online, ‘off-the-shelf’, and for the price of a cup of coffee, shouldn’t be so surprising. It is part of a well-trodden path in cybersecurity.
Trickle-down effect
“Over the last 20 years in cybercrime globally there has been a real trickle-down effect in criminal techniques. What start as highly sophisticated attacks, that take a lot of energy, eventually get industrialised and commoditised, and makes their way down the value chain,” says Simon Brown, Head of Cybersecurity Strategy and Capability at Westpac.
“In the early days of phishing, conducting an attack was a complex exercise. A criminal had to compromise some infrastructure – break into someone’s site – craft their own version of a phishing site, build their own money laundering network, and then put together each of the pieces. Over time, we saw the specialisation of criminal markets, effectively commoditising each of those pieces as criminal products and services.”
In 2020, as the corporate world was forced into adopting ever more virtual and remote ways of working and engaging with the outside world as a result of COVID-19, the embedded risks in purely digital interactions became more pertinent.
“Many organisations’ digital processes today assume a physical interaction at the start – the customer visits the organisation and presents a driver’s licence or similar, and that forms the basis for the trust between customer and organisation. As organisations try to work out how to establish trust with customers in a purely digital environment, new methods bring with them new risks. In particular, if an organisation relies on a video conversation to replace that initial physical interaction and establish trust, then deepfakes become a relevant technique for a criminal looking to subvert that trust,” says Brown.
This hypothetical threat may be all too close, Patrini believes. “Many [organisations] have begun to identify users, or onboard new customers, completely online. You just need an ID … a selfie or a short video where you say something and move your head.
“The threat that biometric companies are considering is where an attacker presents fake ID documents and appears on video with the face of somebody else. Commercial biometric systems, including liveness detectors, can be tricked by this attack. We know this because we have penetrated those systems on multiple occasions.”
Brown says organisations are investing heavily in controls and processes, including hybrid approaches using both humans and computers working together, in anticipation of deceptive technology crossing a threshold of efficacy and ease of use.
“In a hybrid model, the humans can focus on being helpful, while the systems around them are working in the background to detect attempted fraud. Humans are great at wanting to be helpful. So, you want to wrap suspicious systems around helpful humans.”
Smart defence
One challenge is that time is on the villains’ side. “Playing defence is hard,” says Farid. “Here’s a nightmare situation for your business leaders. I could make a video of a CEO saying profits are down, and post it on the internet, moving the market by billions. I can short the stock and be out before anybody realises what’s happened. I can create it and distribute it, with no checks and balances.”
Such checks and balances were a given in the days of traditional media. Today, more than half of Americans get their news from Facebook, according to the Pew Research Centre[1], while many ‘news’ stories that reach the mainstream break first on social media. As a result, traditional media providers have had to become expert at detecting manipulated content.
“Like many news organisations, we have developed a strong specialism in content verification,” says Nick Cohen, Global Head of Product, Core News Services at Reuters. Cohen says that news agencies will typically undertake a variety of measures to ensure veracity, including a reverse image search to see if the material had been posted before, differently, as well as checking geo-location and meta-data. Crucially, they will also seek corroborating imagery, from different cameras for example.
“Our differentiator at Reuters is that our team will directly question the source of the material, just like any other fact-checking activity, which involves actually contacting the media creator,” says Cohen. ”Also, we have thousands of journalists around the world, with different language specialisms, deep subject knowledge, and they are an invaluable resource for the verification team.”
Cohen believes that the experience of news agencies, which are on the frontline of media verification, may be instructive for the corporate sector, and Reuters already works with companies to fact-check and verify social media material. In addition, increasingly, there are technology tools to help.
One example is control capture technology from companies such as TruePic, which verifies media at the point of capture by cryptographically signing the media’s data, time, geo-location and pixels, so any changes will be contained in the media itself. In this way, the original media takes on the responsibility of self-verification, rather than every single recipient having to do so individually.
In addition, the machine learning technologies that make deepfakes possible can also be used to combat the phenomenon. For instance, one emerging technique is to apply ‘adversarial noise’, for example to media of senior executives, as a kind of interference that is undetectable to viewers, but fools face-capture technology, and so helps safeguard the likeness of a public figure from machine learning algorithms that could use it to create synthesised media.
Society, and corporate security teams, are up to the challenge presented by this latest wave of technology-powered deceit, Brown believes.
“The world talks a lot about what the bad guys do – in part because of the work to raise everyone’s awareness. Organisations don’t talk as much, or as publicly, about what organisations and their customers do to prevent attacks from being successful. The engineering work of securing an environment and protecting its customers is nowhere near as newsworthy. But there is far more capability to defend than is commonly talked about.”
Simply being aware of such threats is a significant deterrent, says Brown. “The key thing will be ensuring that the trust people place in a particular channel is not higher than that channel deserves. The things we have told people – about not clicking links in emails and not believing things that are too good to be true – still apply, and we need to collectively update that awareness for new technologies, so people can be personally resilient.”
As with all high-tech threats, people still play an important role. “There is no technology that I’ve ever seen that is as adaptive as a human paying attention,” says Brown. “Technology is great at responding in a deterministic way, and we all build ever more complicated versions of that logic. But there’s no machine as flexible or sophisticated as an alert human.”
Stay informed with Westpac IQ
Get the latest reports straight to your inbox.
Related articles
Business scams on the rise – be alert!
Beating the scammers
Browse topics
Disclaimer
©2025 Westpac Banking Corporation ABN 33 007 457 141 (including where acting under any of its Westpac, St George, Bank of Melbourne or BankSA brands, collectively, “Westpac”). References to the “Westpac Group” are to Westpac and its subsidiaries and includes the directors, employees and representatives of Westpac and its subsidiaries.
Things you should know
We respect your privacy: You can view our privacy statement at Westpac.com.au. Each time someone visits our site, data is captured so that we can accurately evaluate the quality of our content and make improvements for you. We may at times use technology to capture data about you to help us to better understand you and your needs, including potentially for the purposes of assessing your individual reading habits and interests to allow us to provide suggestions regarding other reading material which may be suitable for you.
This information, unless specifically indicated otherwise, is under copyright of the Westpac Group. None of the material, nor its contents, nor any copy of it, may be altered in any way, transmitted to, copied of distributed to any other party without the prior written permission of the Westpac Group.
Disclaimer
This information has been prepared by the Westpac and is intended for information purposes only. It is not intended to reflect any recommendation or financial advice and investment decisions should not be based on it. This information does not constitute an offer, a solicitation of an offer, or an inducement to subscribe for, purchase or sell any financial instrument or to enter into a legally binding contract. To the extent that this information contains any general advice, it has been prepared without taking into account your objectives, financial situation or needs and before acting on it you should consider the appropriateness of the advice. Certain types of transactions, including those involving futures, options and high yield securities give rise to substantial risk and are not suitable for all investors. We recommend that you seek your own independent legal or financial advice before proceeding with any investment decision. This information may contain material provided by third parties. While such material is published with the necessary permission none of Westpac or its related entities accepts any responsibility for the accuracy or completeness of any such material. Although we have made every effort to ensure this information is free from error, none of Westpac or its related entities warrants the accuracy, adequacy or completeness of this information, or otherwise endorses it in any way. Except where contrary to law, Westpac Group intend by this notice to exclude liability for this information. This information is subject to change without notice and none of Westpac or its related entities is under any obligation to update this information or correct any inaccuracy which may become apparent at a later date. This information may contain or incorporate by reference forward-looking statements. The words “believe”, “anticipate”, “expect”, “intend”, “plan”, “predict”, “continue”, “assume”, “positioned”, “may”, “will”, “should”, “shall”, “risk” and other similar expressions that are predictions of or indicate future events and future trends identify forward-looking statements. These forward-looking statements include all matters that are not historical facts. Past performance is not a reliable indicator of future performance, nor are forecasts of future performance. Whilst every effort has been taken to ensure that the assumptions on which any forecasts are based are reasonable, the forecasts may be affected by incorrect assumptions or by known or unknown risks and uncertainties. The ultimate outcomes may differ substantially from any forecasts.
Conflicts of Interest: In the normal course of offering banking products and services to its clients, the Westpac Group may act in several capacities (including issuer, market maker, underwriter, distributor, swap counterparty and calculation agent) simultaneously with respect to a financial instrument, giving rise to potential conflicts of interest which may impact the performance of a financial instrument. The Westpac Group may at any time transact or hold a position (including hedging and trading positions) for its own account or the account of a client in any financial instrument which may impact the performance of that financial instrument.
Author(s) disclaimer and declaration: The author(s) confirms that (a) no part of his/her compensation was, is, or will be, directly or indirectly, related to any views or (if applicable) recommendations expressed in this material; (b) this material accurately reflects his/her personal views about the financial products, companies or issuers (if applicable) and is based on sources reasonably believed to be reliable and accurate; (c) to the best of the author’s knowledge, they are not in receipt of inside information and this material does not contain inside information; and (d) no other part of the Westpac Group has made any attempt to influence this material.
Further important information regarding sustainability-related content: This material may contain statements relating to environmental, social and governance (ESG) topics. These are subject to known and unknown risks, and there are significant uncertainties, limitations, risks and assumptions in the metrics, modelling, data, scenarios, reporting and analysis on which the statements rely. In particular, these areas are rapidly evolving and maturing, and there are variations in approaches and common standards and practice, as well as uncertainty around future related policy and legislation. Some material may include information derived from publicly available sources that have not been independently verified. No representation or warranty is made as to the accuracy, completeness or reliability of the information. There is a risk that the analysis, estimates, judgements, assumptions, views, models, scenarios or projections used may turn out to be incorrect. These risks may cause actual outcomes to differ materially from those expressed or implied. The ESG-related statements in this material do not constitute advice, nor are they guarantees or predictions of future performance, and Westpac gives no representation, warranty or assurance (including as to the quality, accuracy or completeness of the statements). You should seek your own independent advice.
Additional country disclosures:
Australia: Westpac holds an Australian Financial Services Licence (No. 233714). You can access Westpac’s Financial Services Guide here or request a copy from your Westpac point of contact. To the extent that this information contains any general advice, it has been prepared without taking into account your objectives, financial situation or needs and before acting on it you should consider the appropriateness of the advice.
New Zealand: In New Zealand, Westpac Institutional Bank refers to the brand under which products and services are provided by either Westpac (NZ division) or Westpac New Zealand Limited (company number 1763882), the New Zealand incorporated subsidiary of Westpac ("WNZL"). Any product or service made available by WNZL does not represent an offer from Westpac or any of its subsidiaries (other than WNZL). Neither Westpac nor its other subsidiaries guarantee or otherwise support the performance of WNZL in respect of any such product. WNZL is not an authorised deposit-taking institution for the purposes of Australian prudential standards. The current disclosure statements for the New Zealand branch of Westpac and WNZL can be obtained at the internet address www.westpac.co.nz .
Singapore: This material has been prepared and issued for distribution in Singapore to institutional investors, accredited investors and expert investors (as defined in the applicable Singapore laws and regulations) only. Recipients of this material in Singapore should contact Westpac Singapore Branch in respect of any matters arising from, or in connection with, this material. Westpac Singapore Branch holds a wholesale banking licence and is subject to supervision by the Monetary Authority of Singapore.
U.S.: Westpac operates in the United States of America as a federally licensed branch, regulated by the Office of the Comptroller of the Currency. Westpac is also registered with the US Commodity Futures Trading Commission (“CFTC”) as a Swap Dealer, but is neither registered as, or affiliated with, a Futures Commission Merchant registered with the US CFTC. The services and products referenced above are not insured by the Federal Deposit Insurance Corporation (“FDIC”). Westpac Capital Markets, LLC (‘WCM’), a wholly-owned subsidiary of Westpac, is a broker-dealer registered under the U.S. Securities Exchange Act of 1934 (‘the Exchange Act’) and member of the Financial Industry Regulatory Authority (‘FINRA’). In accordance with APRA's Prudential Standard 222 'Association with Related Entities', Westpac does not stand behind WCM other than as provided for in certain legal agreements between Westpac and WCM andobligations of WCM do not represent liabilities of Westpac. This communication is provided for distribution to U.S. institutional investors in reliance on the exemption from registration provided by Rule 15a-6 under the Exchange Act and is not subject to all of the independence and disclosure standards applicable to debt research reports prepared for retail investors in the United States. WCM is the U.S. distributor of this communication and accepts responsibility for the contents of this communication. Transactions by U.S. customers of any securities referenced herein should be effected through WCM. All disclaimers set out with respect to Westpac apply equally to WCM. If you would like to speak to someone regarding any security mentioned herein, please contact WCM on +1 212 389 1269. Investing in any non-U.S. securities or related financial instruments mentioned in this communication may present certain risks. The securities of non-U.S. issuers may not be registered with, or be subject to the regulations of, the SEC in the United States. Information on such non-U.S. securities or related financial instruments may be limited. Non-U.S. companies may not be subject to audit and reporting standards and regulatory requirements comparable to those in effect in the United States. The value of any investment or income from any securities or related derivative instruments denominated in a currency other than U.S. dollars is subject to exchange rate fluctuations that may have a positive or adverse effect on the value of or income from such securities or related derivative instruments.
The author of this communication is employed by Westpac and is not registered or qualified as a research analyst, representative, or associated person of WCM or any other U.S. broker-dealer under the rules of FINRA, any other U.S. self-regulatory organisation, or the laws, rules or regulations of any State. Unless otherwise specifically stated, the views expressed herein are solely those of the author and may differ from the information, views or analysis expressed by Westpac and/or its affiliates.
UK and EU: The London branch of Westpac is authorised in the United Kingdom by the Prudential Regulation Authority (PRA) and is subject to regulation by the Financial Conduct Authority (FCA) and limited regulation by the PRA (Financial Services Register number: 124586). The London branch of Westpac is registered at Companies House as a branch established in the United Kingdom (Branch No. BR000106). Details about the extent of the regulation of Westpac’s London branch by the PRA are available from us on request.
Westpac Europe GmbH (“WEG”) is authorised in Germany by the Federal Financial Supervision Authority (‘BaFin’) and subject to its regulation. WEG’s supervisory authorities are BaFin and the German Federal Bank (‘Deutsche Bundesbank’). WEG is registered with the commercial register (‘Handelsregister’) of the local court of Frankfurt am Main under registration number HRB 118483. In accordance with APRA’s Prudential Standard 222 ‘Association with Related Entities’, Westpac does not stand behind WEG other than as provided for in certain legal agreements (a risk transfer, sub-participation and collateral agreement) between Westpac and WEG and obligations of WEG do not represent liabilities of Westpac.
This communication is not intended for distribution to, or use by any person or entity in any jurisdiction or country where such distribution or use would be contrary to local law or regulation. This communication is not being made to or distributed to, and must not be passed on to, the general public in the United Kingdom. Rather, this communication is being made only to and is directed at (a) those persons falling within the definition of Investment Professionals (set out in Article 19(5) of the Financial Services and Markets Act 2000 (Financial Promotion) Order 2005 (the “Order”)); (b) those persons falling within the definition of high net worth companies, unincorporated associations etc. (set out in Article 49(2)of the Order; (c) other persons to whom it may lawfully be communicated in accordance with the Order or (d) any persons to whom it may otherwise lawfully be made (all such persons together being referred to as “relevant persons”). Any person who is not a relevant person should not act or rely on this communication or any of its contents. In the same way, the information contained in this communication is intended for “eligible counterparties” and “professional clients” as defined by the rules of the Financial Conduct Authority and is not intended for “retail clients”. Westpac expressly prohibits you from passing on the information in this communication to any third party.
This communication contains general commentary, research, and market colour. The communication does not constitute investment advice. The material may contain an ‘investment recommendation’ and/or ‘information recommending or suggesting an investment’, both as defined in Regulation (EU) No 596/2014 (including as applicable in the United Kingdom) (“MAR”). In accordance with the relevant provisions of MAR, reasonable care has been taken to ensure that the material has been objectively presented and that interests or conflicts of interest of the sender concerning the financial instruments to which that information relates have been disclosed.
Investment recommendations must be read alongside the specific disclosure which accompanies them and the general disclosure which can be found here. Such disclosure fulfils certain additional information requirements of MAR and associated delegated legislation and by accepting this communication you acknowledge that you are aware of the existence of such additional disclosure and its contents.
To the extent this communication comprises an investment recommendation it is classified as non-independent research. It has not been prepared in accordance with legal requirements designed to promote the independence of investment research and therefore constitutes a marketing communication. Further, this communication is not subject to any prohibition on dealing ahead of the dissemination of investment research.