Tute Genomics was an American genomics startup that provided a cloud-based web application for rapid and accurate annotation of human genomic data. It was built on the expertise of ANNOVAR. Tute Genomics assisted researchers in identifying disease genes and biomarkers, and assisted clinicians/labs in performing genetic diagnosis. Based in Provo, Utah, Tute was co-founded by Dr. Kai Wang, an assistant professor at the University of Southern California (USC); and Dr. Reid J. Robison, a board-certified psychiatrist with fellowship training in both neurodevelopmental genetics and bioinformatics. Tute Genomics was acquired by PierianDX in 2016. == History == The word "tute" means "personal" in the Na’vi language created for the 2009 film Avatar by Paul Frommer, a linguist and communications professor at the USC Marshall School of Business. === Timeline === 2013 Tute Genomics launched in 2013 and entered the accelerator, BoomStartup. By "demo day" of BoomStartup, Tute had raised their seed round of funding and expanded the round to include angel investors from SLC Angels, Park City Angels, Life Science Angels. Tute was the tenth ever online syndicate for AngelList and in all raised a seed round of $1.5 million. 2014 In March 2014, the company announced that Affiliated Genetics, a Utah-based CLIA-certified laboratory, selected Tute Genomics for its next-generation sequencing (NGS) analytics pipeline. In May 2014, the company announced joining the Global Alliance for Genomics and Health. In June 2014, Advanced Biological Laboratories (ABL), S.A., announced a licensing and collaboration agreement with Tute Genomics and the commercial launch of OncoChek for managing and analysing genomics data in the field of oncology. In July 2014, the company announced an agreement with Lineagen, Inc., to provide next-generation sequencing analytics for Lineagen’s NextStepDx Plus assay. Also, Brigham Young University selected the Tute Genomics genome annotation and discovery platform for analysis and interpretation of 1,000 exomes and genomes. In November 2014, the company announced addition of the Tute platform to Illumina’s BaseSpace. The company announced a Series A1 funding round of $2.3 million in December 2014. The round was led by UK-based Eurovestech. Peak Ventures and a number of angel investors also participated in this round. 2015 Tute recruits David Mittelman, founder of Arpeggi, Inc. and former CSO at FamilyTreeDNA, to Tute Genomics as Chief Scientific Officer. Tute acquires Knome and integrates the KnoSys platform into its software product. 2016 Reid Robison, Tute CEO, launches a Kickstarter campaign to sell Tute interpreted whole genome and whole exome sequencing directly to consumers. The campaign was suspended within the same month after receiving a letter from the United States Food and Drug Administration. Tute is acquired by PierianDX.
Grammar induction
Grammar induction (or grammatical inference) is the process in machine learning of learning a formal grammar (usually as a collection of re-write rules or productions or alternatively as a finite-state machine or automaton of some kind) from a set of observations, thus constructing a model which accounts for the characteristics of the observed objects. More generally, grammatical inference is that branch of machine learning where the instance space consists of discrete combinatorial objects such as strings, trees and graphs. == Grammar classes == Grammatical inference has often been very focused on the problem of learning finite-state machines of various types (see the article Induction of regular languages for details on these approaches), since there have been efficient algorithms for this problem since the 1980s. Since the beginning of the century, these approaches have been extended to the problem of inference of context-free grammars and richer formalisms, such as multiple context-free grammars and parallel multiple context-free grammars. Other classes of grammars for which grammatical inference has been studied are combinatory categorial grammars, stochastic context-free grammars, contextual grammars and pattern languages. == Learning models == The simplest form of learning is where the learning algorithm merely receives a set of examples drawn from the language in question: the aim is to learn the language from examples of it (and, rarely, from counter-examples, that is, example that do not belong to the language). However, other learning models have been studied. One frequently studied alternative is the case where the learner can ask membership queries as in the exact query learning model or minimally adequate teacher model introduced by Angluin. == Methodologies == There is a wide variety of methods for grammatical inference. Two of the classic sources are Fu (1977) and Fu (1982). Duda, Hart & Stork (2001) also devote a brief section to the problem, and cite a number of references. The basic trial-and-error method they present is discussed below. For approaches to infer subclasses of regular languages in particular, see Induction of regular languages. A more recent textbook is de la Higuera (2010), which covers the theory of grammatical inference of regular languages and finite state automata. D'Ulizia, Ferri and Grifoni provide a survey that explores grammatical inference methods for natural languages. === Induction of probabilistic grammars === There are several methods for induction of probabilistic context-free grammars. === Grammatical inference by trial-and-error === The method proposed in Section 8.7 of Duda, Hart & Stork (2001) suggests successively guessing grammar rules (productions) and testing them against positive and negative observations. The rule set is expanded so as to be able to generate each positive example, but if a given rule set also generates a negative example, it must be discarded. This particular approach can be characterized as "hypothesis testing" and bears some similarity to Mitchel's version space algorithm. The Duda, Hart & Stork (2001) text provide a simple example which nicely illustrates the process, but the feasibility of such an unguided trial-and-error approach for more substantial problems is dubious. === Grammatical inference by genetic algorithms === Grammatical induction using evolutionary algorithms is the process of evolving a representation of the grammar of a target language through some evolutionary process. Formal grammars can easily be represented as tree structures of production rules that can be subjected to evolutionary operators. Algorithms of this sort stem from the genetic programming paradigm pioneered by John Koza. Other early work on simple formal languages used the binary string representation of genetic algorithms, but the inherently hierarchical structure of grammars couched in the EBNF language made trees a more flexible approach. Koza represented Lisp programs as trees. He was able to find analogues to the genetic operators within the standard set of tree operators. For example, swapping sub-trees is equivalent to the corresponding process of genetic crossover, where sub-strings of a genetic code are transplanted into an individual of the next generation. Fitness is measured by scoring the output from the functions of the Lisp code. Similar analogues between the tree structured lisp representation and the representation of grammars as trees, made the application of genetic programming techniques possible for grammar induction. In the case of grammar induction, the transplantation of sub-trees corresponds to the swapping of production rules that enable the parsing of phrases from some language. The fitness operator for the grammar is based upon some measure of how well it performed in parsing some group of sentences from the target language. In a tree representation of a grammar, a terminal symbol of a production rule corresponds to a leaf node of the tree. Its parent nodes corresponds to a non-terminal symbol (e.g. a noun phrase or a verb phrase) in the rule set. Ultimately, the root node might correspond to a sentence non-terminal. === Grammatical inference by greedy algorithms === Like all greedy algorithms, greedy grammar inference algorithms make, in iterative manner, decisions that seem to be the best at that stage. The decisions made usually deal with things like the creation of new rules, the removal of existing rules, the choice of a rule to be applied or the merging of some existing rules. Because there are several ways to define 'the stage' and 'the best', there are also several greedy grammar inference algorithms. These context-free grammar generating algorithms make the decision after every read symbol: Lempel-Ziv-Welch algorithm creates a context-free grammar in a deterministic way such that it is necessary to store only the start rule of the generated grammar. Sequitur and its modifications. These context-free grammar generating algorithms first read the whole given symbol-sequence and then start to make decisions: Byte pair encoding and its optimizations. === Distributional learning === A more recent approach is based on distributional learning. Algorithms using these approaches have been applied to learning context-free grammars and mildly context-sensitive languages and have been proven to be correct and efficient for large subclasses of these grammars. === Learning of pattern languages === Angluin defines a pattern to be "a string of constant symbols from Σ and variable symbols from a disjoint set". The language of such a pattern is the set of all its nonempty ground instances i.e. all strings resulting from consistent replacement of its variable symbols by nonempty strings of constant symbols. A pattern is called descriptive for a finite input set of strings if its language is minimal (with respect to set inclusion) among all pattern languages subsuming the input set. Angluin gives a polynomial algorithm to compute, for a given input string set, all descriptive patterns in one variable x. To this end, she builds an automaton representing all possibly relevant patterns; using sophisticated arguments about word lengths, which rely on x being the only variable, the state count can be drastically reduced. Erlebach et al. give a more efficient version of Angluin's pattern learning algorithm, as well as a parallelized version. Arimura et al. show that a language class obtained from limited unions of patterns can be learned in polynomial time. === Pattern theory === Pattern theory, formulated by Ulf Grenander, is a mathematical formalism to describe knowledge of the world as patterns. It differs from other approaches to artificial intelligence in that it does not begin by prescribing algorithms and machinery to recognize and classify patterns; rather, it prescribes a vocabulary to articulate and recast the pattern concepts in precise language. In addition to the new algebraic vocabulary, its statistical approach was novel in its aim to: Identify the hidden variables of a data set using real world data rather than artificial stimuli, which was commonplace at the time. Formulate prior distributions for hidden variables and models for the observed variables that form the vertices of a Gibbs-like graph. Study the randomness and variability of these graphs. Create the basic classes of stochastic models applied by listing the deformations of the patterns. Synthesize (sample) from the models, not just analyze signals with it. Broad in its mathematical coverage, pattern theory spans algebra and statistics, as well as local topological and global entropic properties. == Applications == The principle of grammar induction has been applied to other aspects of natural language processing, and has been applied (among many other problems) to semantic parsing, natural language understanding, example-based translation, language acquisition, grammar-based compre
StarDict
StarDict, developed by Hu Zheng (胡正), is a free GUI released under the GPL-3.0-or-later license for accessing StarDict dictionary files (a dictionary shell). It is the successor of StarDic, developed by Ma Su'an (馬蘇安), continuing its version numbers. According to StarDict's earlier homepage on SourceForge, the project has been removed from SourceForge due to copyright infringement reports. It moved to Google Code and then back to SourceForge, while development is now seemingly continued on GitHub. == Supported platforms == StarDict runs under Linux, Windows, FreeBSD, Maemo and Solaris. Dictionaries of the user's choice are installed separately. Dictionary files can be created by converting dict files. Several programs compatible with the StarDict dictionary format are available for different platforms. For the iPhone, iPod Touch and iPad, applications available in the App Store include GuruDic, TouchDict, weDict, Dictionary Universal, Alpus and others, as well as the free iStarDict, which is available for the Cydia Store. == Dictionaries available == One can find here the partial list of FreeDict dictionaries which can be converted to the StarDict format. These include, in particular, some older versions of Webster's dictionary and many dictionaries for various languages. == Features == While StarDict is in scan mode, results are displayed in a tooltip, allowing easy dictionary lookup. When combined with Freedict, StarDict will quickly provide rough translations of foreign language websites. On September 25, 2006, an online version of Stardict began operation. This online version includes access to all the major dictionaries of StarDict, as well as Wikipedia in Chinese. Previous versions of StarDict were very similar to the PowerWord dictionary program, which is developed by a Chinese company, KingSoft. Since version 2.4.2, however, StarDict has diverged from the design of PowerWord by increasing its search capabilities and adding lexicons in a variety of languages. This was assisted by the collaboration of many developers with the author. == sdcv == Evgeniy A. Dushistov produced a command line version of StarDict called sdcv. It employed all the dictionary files that belong to StarDict. It is written in C++ and licensed under the terms of the GNU General Public License. sdcv runs under Linux, FreeBSD, and Solaris. As in StarDict, dictionaries of the user's choice have to be installed separately. At the end of 2006, software developer Hu Zheng cited personal financial problems as an excuse to charge users for downloading dictionary files from his website, which temporarily aroused strong doubts and dissatisfaction in the Linux community. In the end, under the pressure of public opinion, the charging plan was forced to be canceled and ended hastily.
Salvatore J. Stolfo
Salvatore J. Stolfo is an academic and professor of computer science at Columbia University, specializing in computer security. == Early life == Born in Brooklyn, New York, Stolfo received a Bachelor of Science degree in Computer Science and Mathematics from Brooklyn College in 1974. He received his Ph.D. from NYU Courant Institute in 1979 and has been on the faculty of Columbia ever since, where he's taught courses in Artificial Intelligence, Intrusion and Anomaly Detection Systems, Introduction to Programming, Fundamental Algorithms, Data Structures, and Knowledge-Based Expert Systems. == Academic research == While at Columbia, Stolfo has received close to $50M in funding for research that has broadly focused on Security, Intrusion Detection, Anomaly Detection, Machine Learning and includes early work in parallel computing and artificial intelligence. He has published or co-authored over 250 papers and has over 46,000 citations with an H-index of 102. In 1996 he proposed a project with DARPA that applies machine learning to behavioral patterns to detect fraud or intrusion in networks. DADO, developed by in part by Stolfo, introduced the parallel computing primitive: “Broadcast, Resolve, Report”, a hardwire implemented mechanism that today is called MapReduce. Among his earliest work, Stolfo along with colleague Greg Vesonder of Bell Labs, developed a large-scale expert data analysis system, called ACE (Automated Cable Expertise) for the nation's phone system. AT&T Bell Labs distributed ACE to a number of telephone wire centers to improve the management and scheduling of repairs in the local loop. Stolfo coined the term FOG computing (not to be confused with fog computing) where technology is used “to launch disinformation attacks against malicious insiders, preventing them from distinguishing the real sensitive customer data from fake worthless data.” In 2005 Stolfo received funding from the Army Research Office to conduct a workshop to bring together a group of researchers to help identify a research program to focus on insider threats. He was elevated to IEEE Fellow in 2018 "for his contributions to machine learning based cybersecurity." He was elected as an ACM Fellow in 2019 "for contributions to machine-learning-based cybersecurity and parallel hardware for database inference systems". == Career == Founded in 2011, Red Balloon Security (or RBS) is a cyber security company founded by Dr Sal Stolfo and Dr Ang Cui. A spinout from the IDS lab, RBS developed a symbiote technology called FRAK as a host defense for embedded systems under the sponsorship of DARPA's Cyber Fast Track program. Created based on their IDS lab research for the DARPA Active Authentication and the Anomaly Detection at Multiple Scales program, Dr Sal Stolfo and Dr. Angelos Keromytis founded Allure Security Technologies. Using active behavioral authentication and decoy technology Stolfo pioneered and patented in 1996. Founded in 2009, Allure Security Technology was created based on work done under DARPA sponsorship in Columbia's IDS lab based on DARPA prompts to research how to detect hackers once they are inside an organization's perimeter and how to continuously authenticate a user without a password. Stolfo's company Electronic Digital Documents produced a “DataBlade” technology, which Informix marketed during their strategy of acquisition and development in the mid 80's. Stolfo's patented merge/purge technology called EDD DataCleanser DataBlade was licensed by Informix. Since its acquisition by IBM in 2005, IBM Informix is one of the world's most widely used database servers, with users ranging from the world's largest corporations to startups. System Detection was one of the companies founded by Prof. Stolfo to commercialize the Anomaly Detection technology developed in the IDS lab. The company ultimately reorganized and was rebranded as Trusted Computer Solutions. That company was recently acquired by Raytheon. Recently a jury awarded Columbia University $185 million for patent infringement for one of Prof. Stolfo's inventions, the Application Communities technology. https://news.columbia.edu/news/columbia-university-awarded-185-million-patent-infringement-nortonlifelock-inc. The final order from the judge applied nearly treble damages: https://www.reuters.com/legal/litigation/gen-digital-owes-columbia-481-mln-us-patent-fight-judge-says-2023-10-02/
IBM optical mark and character readers
IBM designed, manufactured and sold optical mark and character readers from 1960 until 1984. The IBM 1287 is notable as being the first commercially sold scanner capable of reading handwritten numbers. == Initial development work == IBM Poughkeepsie studied machine character recognition from 1950 till 1954, developing an experimental machine that used a cathode-ray-tube attached an IBM 701 which performed the character analysis. They pursued a technique known as lakes and bays which examined different areas of dark and light where the lakes were white areas enclosed by black and the bays were partially enclosed areas. Their machine and mission was moved to IBM Endicott in 1954, where research continued. From 1955 to 1956 they then worked on the VIDOR (Visual Document Reader) program, but they could not get agreement on acceptable reject rate. The developers felt 80% recognition was acceptable (meaning 20% of documents would need to be manually processed), while product planners and IBM Marketing felt that compared to punched card, the reject rate was unacceptably high. This led to no new products being released. In 1956 the American Bankers Association chose to use Magnetic Ink Character Recognition (MICR) to automate check handling, rejecting a proposed solution generated by an IBM Poughkeepsie banking project that used optical characters formed by vertical bars and digits. IBM developed a magnetic read head to handle the new standard, releasing the IBM 1210 MICR reader/sorter in 1959. The development work for this product both with read heads and document handling, helped move optical character recognition forward, with development focusing on reading one or two lines of print from a paper document larger than an IBM punched card. The first product to be released was the IBM 1418. == IBM 123x Optical Mark Readers == The IBM 1230, IBM 1231, and IBM 1232 were optical mark readers used to input the contents of data sources such as questionnaires, test results, surveys as well as historical data that could be easily entered as marks on sheets. Educational institutes used them to score test results and they were effectively a replacement for the IBM 805 Test Scoring Machine that used electrical resistance and a mark sense pencil to score a test, rather than optical mark detection. They were developed and manufactured by IBM Rochester. They have the following features: A pneumatic input hopper that can hold approximately 600 sheets Two output stackers: the normal stacker that holds 600 sheets and the select (or reject) stacker which holds 50 sheets. Pluggable SMS printed circuit cards They can read positional marks made by a lead pencil using an optical read head that consists of photovoltaic(solar) cells and lamps The 1230 has 21 photovoltaic cells, 20 for reading the pencil marks and one to read timing marks on the right hand border of the sheet. The 1231 and 1232 have 22 photovoltaic cells, 20 to read data, one to read timing marks and one to read a special feature called a master mark. Input size is a 8+1⁄2 in × 11 in (22 cm × 28 cm) sheet called a data sheet that can have up to 1000 marked or printed positions per side. Uses electromechanical devices known as sonic delay lines to store results. === IBM 1230 Optical Mark Scoring Reader === The IBM 1230 is an offline optical mark scoring machine announced on 2 November 1962 that was designed to read and scores 1,200 answer sheets per hour. Scored results are printed via a wire matrix printer on the right margin of each answer sheet as it is processed. Two master sheets are required for the process: one that encoded the correct answers and one for the machine to record run information. Output could be sent to an IBM 534 Model 3 Card Punch as an option, which limits throughput to 750 sheets per hour when punching 80 columns of data. === IBM 1231 Optical Mark Page Reader === The IBM 1231 is an online optical mark reader that was designed to read and score 2000 test answer sheets per hour, depending on downstream operations. The correct answers for the test can either be entered using a master sheet (like the 1230) or sent to the 1231 using the optional master-mark special feature. === IBM 1232 Optical Mark Page Reader === The IBM 1232 is an offline optical mark reader that was designed to read up to 2000 marked sheets per hour. Documents can be read at up to 2000 sheets per hour, but this depends on the number of characters that need to be punched from each sheet. The IBM 1232 reads the marks and then punches them into cards using a IBM 534 Model 3 Card Punch. Together they can read up to 64,000 characters per hour or 800 fully punched cards. === Example customers === The California Test Bureau (CTB) that provided standardised achievement tests for educational institutes across the USA, began replacing their IBM 805s with IBM 1230s in 1963. They then installed two IBM 1232s in 1964. Being able to use a full 8+1⁄2 in × 11 in (22 cm × 28 cm) answer sheet rather than a 7+3⁄8 in × 3+1⁄4 in (18.7 cm × 8.3 cm) mark sense card, eliminated the need to use multiple answer cards per test per student, as well as dramatically increased the marking speed for test answers. Credit Bureau Services of Dallas used an IBM 1232 in 1966 as part of their first computerisation project. They marked credit history data onto optical scanning sheets that were fed into their IBM 1232. The attached IBM 534 then punched this data onto punched cards, which were then fed into their IBM System/360 Model 30. In 1968 the US Army Corps of Engineers Coastal Engineering Research Center (CERC) began using special log books for their coastal surveyors to record coastal survey data, which was then converted to punched cards by an IBM 1232. == IBM 2956 Optical Mark/Hole Reader == The IBM 2956 Models 2 and 3 are custom build optical mark/hole readers designed to be attached to an IBM 2740 Communications Terminal. The IBM 2956-2 can read cards that have either been hand or machine marked or that have been punched. The cards can be fed by hand or from the 400 card hopper. It has a 400 card stacker. The 2956-2 could be ordered by request for price quotation (RPQ) 843086. The IBM 2956-3 can read cards that have either been hand or machine marked or that have been punched. It can also read marked sheets up to 9 in × 14 in (230 mm × 360 mm) in size, although only a 3+1⁄4 in (83 mm) band along the side of the sheet can be read (the width of a punched card). It does not have a hopper or a stacker, so each card or sheet must be manually fed into the machine. The 2956-3 could be ordered by request for price quotation (RPQ) 843106. The 2956-3 could be attached to an IBM 3276 or IBM 3278 display station with RPQ UB9001. One use case for the IBM 2956 is to grade school tests. On completion of a learning module a student can use an optical scan-type card to record answers to up to 27 questions, with up to 5 choices per question. They are scanned by the reader and the results are then transmitted to an IBM System/360 in remote job entry mode and can also be printed on the IBM 2740. The reader can also be attached to an IBM 3735 which transmits results to an IBM System/370 and which prints results on an IBM 3286 printer. They can also be attached to an IBM System/3. Note that the IBM 2956 Model 5 (2956-5) was a banking reader/sorter. == IBM 1282 Optical Reader Card Punch == The IBM 1282 is an offline optical reader that is used to read embossed credit card receipts, a mark read field or machine printed characters in three different fonts. It then outputs this data onto a punched card. It was developed and manufactured by IBM Endicott. It proved popular and within two years of announcement 100 machines were installed or on order. === Example customer === The New York Department of Motor Vehicles reported that from 1964 until 1968 they were using an IBM 1282 to read machine printed license renewal slips that had been mailed back as part of the renewal process. They would scan the slip and then process the resulting punched card. This worked well until the DMV decided to request renewals include the drivers Social Security Number (SSN), which meant a handwritten number needed to be either manually keyed or a new scanning device procured. They switched to the IBM 1287 in 1968. == IBM 1285 Optical Reader == The IBM 1285 is an online optical reader that is used to read printed paper tapes from cash registers or adding machines. It was developed by IBM Endicott and manufactured by IBM Rochester. The IBM 1285 attaches to an IBM 1401, 1440, 1460 or System/360. It has a small round screen to display characters being read and it has a keyboard to enter header information and to optionally enter character corrections for rejected characters. It can read a 200 ft (61 m) roll or paper tape in three-and-a half minutes, reading data at speeds of up to 3000 lines per minute. It can mark the tape with a dot to indicate unreadable characters, so they can be r
Scale-space axioms
In image processing and computer vision, a scale space framework can be used to represent an image as a family of gradually smoothed images. This framework is very general and a variety of scale space representations exist. A typical approach for choosing a particular type of scale space representation is to establish a set of scale-space axioms, describing basic properties of the desired scale-space representation and often chosen so as to make the representation useful in practical applications. Once established, the axioms narrow the possible scale-space representations to a smaller class, typically with only a few free parameters. A set of standard scale space axioms, discussed below, leads to the linear Gaussian scale-space, which is the most common type of scale space used in image processing and computer vision. == Scale space axioms for the linear scale-space representation == The linear scale space representation L ( x , y , t ) = ( T t f ) ( x , y ) = g ( x , y , t ) ∗ f ( x , y ) {\displaystyle L(x,y,t)=(T_{t}f)(x,y)=g(x,y,t)f(x,y)} of signal f ( x , y ) {\displaystyle f(x,y)} obtained by smoothing with the Gaussian kernel g ( x , y , t ) {\displaystyle g(x,y,t)} satisfies a number of properties 'scale-space axioms' that make it a special form of multi-scale representation: linearity T t ( a f + b h ) = a T t f + b T t h {\displaystyle T_{t}(af+bh)=aT_{t}f+bT_{t}h} where f {\displaystyle f} and h {\displaystyle h} are signals while a {\displaystyle a} and b {\displaystyle b} are constants, shift invariance T t S ( Δ x , Δ y ) f = S ( Δ x , Δ y ) T t f {\displaystyle T_{t}S_{(\Delta x,\Delta _{y})}f=S_{(\Delta x,\Delta _{y})}T_{t}f} where S ( Δ x , Δ y ) {\displaystyle S_{(\Delta x,\Delta _{y})}} denotes the shift (translation) operator ( S ( Δ x , Δ y ) f ) ( x , y ) = f ( x − Δ x , y − Δ y ) {\displaystyle (S_{(\Delta x,\Delta _{y})}f)(x,y)=f(x-\Delta x,y-\Delta y)} semi-group structure g ( x , y , t 1 ) ∗ g ( x , y , t 2 ) = g ( x , y , t 1 + t 2 ) {\displaystyle g(x,y,t_{1})g(x,y,t_{2})=g(x,y,t_{1}+t_{2})} with the associated cascade smoothing property L ( x , y , t 2 ) = g ( x , y , t 2 − t 1 ) ∗ L ( x , y , t 1 ) {\displaystyle L(x,y,t_{2})=g(x,y,t_{2}-t_{1})L(x,y,t_{1})} existence of an infinitesimal generator A {\displaystyle A} ∂ t L ( x , y , t ) = ( A L ) ( x , y , t ) {\displaystyle \partial _{t}L(x,y,t)=(AL)(x,y,t)} non-creation of local extrema (zero-crossings) in one dimension, non-enhancement of local extrema in any number of dimensions ∂ t L ( x , y , t ) ≤ 0 {\displaystyle \partial _{t}L(x,y,t)\leq 0} at spatial maxima and ∂ t L ( x , y , t ) ≥ 0 {\displaystyle \partial _{t}L(x,y,t)\geq 0} at spatial minima, rotational symmetry g ( x , y , t ) = h ( x 2 + y 2 , t ) {\displaystyle g(x,y,t)=h(x^{2}+y^{2},t)} for some function h {\displaystyle h} , scale invariance g ^ ( ω x , ω y , t ) = h ^ ( ω x φ ( t ) , ω x φ ( t ) ) {\displaystyle {\hat {g}}(\omega _{x},\omega _{y},t)={\hat {h}}({\frac {\omega _{x}}{\varphi (t)}},{\frac {\omega _{x}}{\varphi (t)}})} for some functions φ {\displaystyle \varphi } and h ^ {\displaystyle {\hat {h}}} where g ^ {\displaystyle {\hat {g}}} denotes the Fourier transform of g {\displaystyle g} , positivity g ( x , y , t ) ≥ 0 {\displaystyle g(x,y,t)\geq 0} , normalization ∫ x = − ∞ ∞ ∫ y = − ∞ ∞ g ( x , y , t ) d x d y = 1 {\displaystyle \int _{x=-\infty }^{\infty }\int _{y=-\infty }^{\infty }g(x,y,t)\,dx\,dy=1} . In fact, it can be shown that the Gaussian kernel is a unique choice given several different combinations of subsets of these scale-space axioms: most of the axioms (linearity, shift-invariance, semigroup) correspond to scaling being a semigroup of shift-invariant linear operator, which is satisfied by a number of families integral transforms, while "non-creation of local extrema" for one-dimensional signals or "non-enhancement of local extrema" for higher-dimensional signals are the crucial axioms which relate scale-spaces to smoothing (formally, parabolic partial differential equations), and hence select for the Gaussian. The Gaussian kernel is also separable in Cartesian coordinates, i.e. g ( x , y , t ) = g ( x , t ) g ( y , t ) {\displaystyle g(x,y,t)=g(x,t)\,g(y,t)} . Separability is, however, not counted as a scale-space axiom, since it is a coordinate dependent property related to issues of implementation. In addition, the requirement of separability in combination with rotational symmetry per se fixates the smoothing kernel to be a Gaussian. There exists a generalization of the Gaussian scale-space theory to more general affine and spatio-temporal scale-spaces. In addition to variabilities over scale, which original scale-space theory was designed to handle, this generalized scale-space theory also comprises other types of variabilities, including image deformations caused by viewing variations, approximated by local affine transformations, and relative motions between objects in the world and the observer, approximated by local Galilean transformations. In this theory, rotational symmetry is not imposed as a necessary scale-space axiom and is instead replaced by requirements of affine and/or Galilean covariance. The generalized scale-space theory leads to predictions about receptive field profiles in good qualitative agreement with receptive field profiles measured by cell recordings in biological vision. In the computer vision, image processing and signal processing literature there are many other multi-scale approaches, using wavelets and a variety of other kernels, that do not exploit or require the same requirements as scale space descriptions do; please see the article on related multi-scale approaches. There has also been work on discrete scale-space concepts that carry the scale-space properties over to the discrete domain; see the article on scale space implementation for examples and references.
DALL-E
DALL-E, DALL-E 2, and DALL-E 3 (stylised DALL·E) are text-to-image models developed by OpenAI using deep learning methodologies to generate digital images from natural language descriptions known as prompts. The first version of DALL-E was announced in January 2021. In the following year, its successor DALL-E 2 was released. DALL-E 3 was released natively into ChatGPT for ChatGPT Plus and ChatGPT Enterprise customers in October 2023, with availability via OpenAI's API and "Labs" platform provided in early November. Microsoft implemented the model in Bing's Image Creator tool and plans to implement it into their Designer app. With Bing's Image Creator tool, Microsoft Copilot runs on DALL-E 3. In March 2025, DALL-E-3 was replaced in ChatGPT by GPT Image's native image-generation capabilities. == History and background == DALL-E was revealed by OpenAI in a blog post on 5 January 2021, and uses a version of GPT-3 modified to generate images. On 6 April 2022, OpenAI announced DALL-E 2, a successor designed to generate more realistic images at higher resolutions that "can combine concepts, attributes, and styles". On 20 July 2022, DALL-E 2 entered into a beta phase with invitations sent to 1 million waitlisted individuals; users could generate a certain number of images for free every month and may purchase more. Access had previously been restricted to pre-selected users for a research preview due to concerns about ethics and safety. On 28 September 2022, DALL-E 2 was opened to everyone and the waitlist requirement was removed. In September 2023, OpenAI announced their latest image model, DALL-E 3, capable of understanding "significantly more nuance and detail" than previous iterations. In early November 2022, OpenAI released DALL-E 2 as an API, allowing developers to integrate the model into their own applications. Microsoft unveiled their implementation of DALL-E 2 in their Designer app and Image Creator tool included in Bing and Microsoft Edge. The API operates on a cost-per-image basis, with prices varying depending on image resolution. Volume discounts are available to companies working with OpenAI's enterprise team. The software's name is a portmanteau of the names of animated robot Pixar character WALL-E and the Spanish surrealist artist Salvador Dalí. In February 2024, OpenAI began adding watermarks to DALL-E generated images, containing metadata in the C2PA (Coalition for Content Provenance and Authenticity) standard promoted by the Content Authenticity Initiative. == Technology == The first generative pre-trained transformer (GPT) model was initially developed by OpenAI in 2018, using a Transformer architecture. The first iteration, GPT-1, was scaled up to produce GPT-2 in 2019; in 2020, it was scaled up again to produce GPT-3, with 175 billion parameters. === DALL-E === DALL-E has three components: a discrete VAE, an autoregressive decoder-only Transformer model (12 billion parameters) similar to GPT-3, and a CLIP pair of image encoder and text encoder. The discrete VAE can convert an image to a sequence of tokens, and conversely, convert a sequence of tokens back to an image. This is necessary as the Transformer model does not directly process image data. The input to the Transformer model is a sequence of tokenised image caption followed by tokenised image patches. The image caption is in English, tokenised by byte pair encoding (vocabulary size 16384), and can be up to 256 tokens long. Each image is a 256×256 RGB image, divided into 32×32 patches of 4×4 each. Each patch is then converted by a discrete variational autoencoder to a token (vocabulary size 8192). DALL-E was developed and announced to the public in conjunction with CLIP (Contrastive Language-Image Pre-training). CLIP is a separate model based on contrastive learning that was trained on 400 million pairs of images with text captions scraped from the Internet. Its role is to "understand and rank" DALL-E's output by predicting which caption from a list of 32,768 captions randomly selected from the dataset (of which one was the correct answer) is most appropriate for an image. A trained CLIP pair is used to filter a larger initial list of images generated by DALL-E to select the image that is closest to the text prompt. === DALL-E 2 === DALL-E 2 uses 3.5 billion parameters, a smaller number than its predecessor. Instead of an autoregressive Transformer, DALL-E 2 uses a diffusion model conditioned on CLIP image embeddings, which, during inference, are generated from CLIP text embeddings by a prior model. This is the same architecture as that of Stable Diffusion, released a few months later. === DALL-E 3 === While a technical report was written for DALL-E 3, it does not include training or implementation details of the model, instead focusing on the improved prompt following capabilities developed for DALL-E 3. == Capabilities == DALL-E can generate imagery in multiple styles, including photorealistic imagery, paintings, and emoji. It can "manipulate and rearrange" objects in its images, and can correctly place design elements in novel compositions without explicit instruction. Thom Dunn writing for BoingBoing remarked that "For example, when asked to draw a daikon radish blowing its nose, sipping a latte, or riding a unicycle, DALL-E often draws the handkerchief, hands, and feet in plausible locations." DALL-E showed the ability to "fill in the blanks" to infer appropriate details without specific prompts, such as adding Christmas imagery to prompts commonly associated with the celebration, and appropriately placed shadows to images that did not mention them. Furthermore, DALL-E exhibits a broad understanding of visual and design trends. DALL-E can produce images for a wide variety of arbitrary descriptions from various viewpoints with only rare failures. Mark Riedl, an associate professor at the Georgia Tech School of Interactive Computing, found that DALL-E could blend concepts (described as a key element of human creativity). Its visual reasoning ability is sufficient to solve Raven's Matrices (visual tests often administered to humans to measure intelligence). DALL-E 3 follows complex prompts with more accuracy and detail than its predecessors, and is able to generate more coherent and accurate text. DALL-E 3 is integrated into ChatGPT Plus. === Image modification === Given an existing image, DALL-E 2 and DALL-E 3 can produce "variations" of the image as individual outputs based on the original, as well as edit the image to modify or expand upon it. The "inpainting" and "outpainting" abilities of these models use context from an image to fill in missing areas using a medium consistent with the original, following a given prompt. For example, this can be used to insert a new subject into an image, or expand an image beyond its original borders. According to OpenAI, "Outpainting takes into account the image’s existing visual elements — including shadows, reflections, and textures — to maintain the context of the original image." === Technical limitations === DALL-E 2's language understanding has limits. It is sometimes unable to distinguish "A yellow book and a red vase" from "A red book and a yellow vase" or "A panda making latte art" from "Latte art of a panda". It generates images of an astronaut riding a horse when presented with the prompt "a horse riding an astronaut". It also fails to generate the correct images in a variety of circumstances. Requesting more than three objects, negation, numbers, and connected sentences may result in mistakes, and object features may appear on the wrong object. Additional limitations include generating text, ambigrams and other forms of typography, which often results in dream-like gibberish. The model also has a limited capacity to address scientific information, such as astronomy or medical imagery. == Ethical concerns == DALL-E 2's reliance on public datasets influences its results and leads to algorithmic bias in some cases, such as generating higher numbers of men than women for requests that do not mention gender. DALL-E 2's training data was filtered to remove violent and sexual imagery, but this was found to increase bias in some cases such as reducing the frequency of women being generated. OpenAI hypothesise that this may be because women were more likely to be sexualised in training data which caused the filter to influence results. In September 2022, OpenAI confirmed to The Verge that DALL-E invisibly inserts phrases into user prompts to address bias in results; for instance, "black man" and "Asian woman" are inserted into prompts that do not specify gender or race. OpenAI claims to address concerns for potential "racy content" – containing nudity or sexual content generation, with DALL-E 3 through input/output filters, blocklists, ChatGPT refusals, and model level interventions. However, DALL-E 3 continues to disproportionally represent people as White, female, and youthful. Users are able to somewhat remedy