The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘Algorithms’ Category

How is NSA breaking so much crypto? “weak” standard primes for Diffie-Hellman are being widely used and take NSA only ~$100 million to crack

Posted by jpluimers on 2015/11/19

Interesting: a few quotes below, read How is NSA breaking so much crypto? and the full paper Imperfect Forward Secrecy: How Diffie-Hellman Fails in Practice for details.

The key is, somewhat ironically, Diffie-Hellman key exchange, an algorithm that we and many others have advocated as a defense against mass surveillance. Diffie-Hellman is a cornerstone of modern cryptography used for VPNs, HTTPS websites, email, and many other protocols. Our paper shows that, through a confluence of number theory and bad implementation choices, many real-world users of Diffie-Hellman are likely vulnerable to state-level attackers.

.. there was a very important detail that got lost in translation between the mathematicians and the practitioners: an adversary can perform a single enormous computation to “crack” a particular prime, then easily break any individual connection that uses that prime.

How enormous a computation, you ask? …  For the most common strength of Diffie-Hellman (1024 bits), it would cost a few hundred million dollars to build a machine, based on special purpose hardware, that would be able to crack one Diffie-Hellman prime every year.

Would this be worth it for an intelligence agency? Since a handful of primes are so widely reused, the payoff, in terms of connections they could decrypt, would be enormous. Breaking a single, common 1024-bit prime would allow NSA to passively decrypt connections to two-thirds of VPNs and a quarter of all SSH servers globally. Breaking a second 1024-bit prime would allow passive eavesdropping on connections to nearly 20% of the top million HTTPS websites. In other words, a one-time investment in massive computation would make it possible to eavesdrop on trillions of encrypted connections.

NSA could afford such an investment. The 2013 “black budget” request …  shows that the agency’s budget is on the order of $10 billion a year, with over $1 billion dedicated to computer network exploitation, and several subprograms in the hundreds of millions a year.

… However, our proposed Diffie-Hellman break fits the known technical details about their large-scale decryption capabilities better than any competing explanation. For instance, the Snowden documents show that NSA’s VPN decryption infrastructure involves intercepting encrypted connections and passing certain data to supercomputers, which return the key. The design of the system goes to great lengths to collect particular data that would be necessary for an attack on Diffie-Hellman but not for alternative explanations, like a break in AES or other symmetric crypto.

Since weak use of Diffie-Hellman is widespread in standards and implementations, it will be many years before the problems go away, even given existing security recommendations and our new findings. In the meantime, other large governments potentially can implement similar attacks, if they haven’t already.

Our findings illuminate the tension between NSA’s two missions, gathering intelligence and defending U.S. computer security. If our hypothesis is correct, the agency has been vigorously exploiting weak Diffie-Hellman, while taking only small steps to help fix the problem. On the defensive side, NSA has recommended that implementors should transition to elliptic curve cryptography, which isn’t known to suffer from this loophole, but such recommendations tend to go unheeded absent explicit justifications or demonstrations. This problem is compounded because the security community is hesitant to take NSA recommendations at face value, following apparent efforts to backdoor cryptographic standards.

–jeroen

via:

Posted in Algorithms, Development, Encryption, Power User, Security, Software Development | Leave a Comment »

On Epsilon, MachineEpsilon, and relative differences – via: I was wondering, that what is the closest value to the Zero floating point can have – G+

Posted by jpluimers on 2015/10/07

A long time ago, there was an interesting discussion here: I was wondering, that what is the closest value to the Zero floating point can have.

Recently I needed to do some calculations on series where getting close to zero could become a problem.

  • Math seems to have an Epsilon of 1E-12.
  • Sytem.Types has Epsilon of 1E-30 and Epsilon2 of 1E-40.
  • XE4+ FMX has IsEssentiallyZero and IsNotEssentiallyZero for Single values.

In practice it depends a lot on what you are doing. Sometimes absolute Epsilons are best, but at other times relative difference is much more applicable.

Then there is also a Machine Epsilon: a way to derive an Epsilon from a data type that works in all languages and platforms.

–jeroen

Posted in .NET, Algorithms, C, C#, C++, Delphi, Development, Floating point handling, Software Development | 1 Comment »

Excel: get content of a cell given the row and column numbers (ADDRESS, INDIRECT, ROW, COLUMN)

Posted by jpluimers on 2015/08/28

A while ago, I needed to do calculations on partially absolute cell references: for some number of rows, the cells needed to be fixed to the top row of each row group.

For a pure absolute cell reference, you’d prepend a dollar sign to the row or column of a cell. So A1 would become $A1 (to make column A absolute), A$1 (to make row 1 absolute) or $A$1 (to make both column A and row 1 absolute).

There is a nice short cut function key F4 to do this.

Excel does not have a built-in partially absolute cell reference solution.

To solve this, I used these addressing functions: ADDRESSINDIRECTCOLUMNROW.

For all these functions, the ROW and COLUMN numbering starts at 1 (one) not 0 (zero).

The way I solved it was to added the below columns (first the reference:heading, then the formula).

The values in the formulas are for ROW 2 (cells A2..XFD2).

  1. AF: Calculation
    • =IF(D2=”*”;INDIRECT(AG2)&X2;””)
  2. AG: ZLookup
    • =ADDRESS($AH2;COLUMN(Z2))
  3. AH: Row
    • =2+12*TRUNC((ROW()-2)/12)

Column AH

Calculates the fixed row of the row group. There are 12 rows per group. ROW numbers start at 1, and there is one heading row, hence the 2+ and the -2.

Without TRUNC, the ROW result would be rounded (that is the default floating point to integer conversion that Excel uses).

There is no need to reference a specific row when calling ROW: if you leave it out, it will return the number of the current row.

Column AG

returns the address of the calculated ROW (from AH) combined with the

Column AF

Depending on the value of the D column, it calculates the outcome by combining

–jeroen

via:

Posted in Algorithms, Development, Excel, Floating point handling, Office, Office 2003, Office 2007, Office 2010, Office 2013, Power User, Software Development | 2 Comments »

CRC: The C Stands for Confusion (quotes from the Dr Dobb’s article)

Posted by jpluimers on 2015/04/28

The CRC algorithm isn’t hard, but choosing the right polynomial and parameters can be tricky especially since there are so many permutations to choose from and lots of libraries have made their own (often incompatible) choices.

 A few quotes

Basics:       Read the rest of this entry »

Posted in Algorithms, Development, Software Development | Leave a Comment »

DanielRapp/doppler

Posted by jpluimers on 2015/03/28

Look ma: no mouse! DanielRapp/doppler. Detecting motion with your built-in computer speakers and microphone using the Doppler effect.

Cool!

–jeroen

Posted in Algorithms, Development, Software Development | Leave a Comment »

Default comparers in Delphi used by TArray.Sort (via: Stack Overflow)

Posted by jpluimers on 2014/11/26

A long while ago, I posted a detailed answer on what functions the default comparers actually were calling to get a feel for if they would apply or not answering delphi – What does the default TArray.Sort comparator actually do and when would you use it? – Stack Overflow.

I needed that information recently because of some sorting issues I bumped into (sorting generic records), so finally a blog post.

First some links to documentation for even more background information:

There is the answer I gave: Read the rest of this entry »

Posted in Algorithms, Ansi, Delphi, Delphi 2009, Delphi 2010, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Development, Encoding, Floating point handling, Software Development, Unicode | 2 Comments »

Delphi: mapping of COM/Windows exceptions to Delphi Exceptions and run-time errors

Posted by jpluimers on 2014/07/30

Every time

---------------------------
Debugger Exception Notification
---------------------------
Project Some.exe raised exception class $C0000005 with message 'access violation at 0x00537b14: read of address 0x00000000'.
---------------------------
Break   Continue   Help
---------------------------

'Exception "EAccessViolation" with message "Access violation at address 00537B30 in module ''TestWordInterface.exe''. Read of address 00000000" at address 00537B30'

---------------------------
Testwordinterface
---------------------------
Access violation at address 00537B14 in module 'Some.exe'. Read of address 00000000.
---------------------------
OK
---------------------------

In system.pas:

{$IFDEF STACK_BASED_EXCEPTIONS}
procedure       MapToRunError(P: PExceptionRecord); stdcall;
const
  STATUS_ACCESS_VIOLATION         = $C0000005;
  STATUS_ARRAY_BOUNDS_EXCEEDED    = $C000008C;
  STATUS_FLOAT_DENORMAL_OPERAND   = $C000008D;
  STATUS_FLOAT_DIVIDE_BY_ZERO     = $C000008E;
  STATUS_FLOAT_INEXACT_RESULT     = $C000008F;
  STATUS_FLOAT_INVALID_OPERATION  = $C0000090;
  STATUS_FLOAT_OVERFLOW           = $C0000091;
  STATUS_FLOAT_STACK_CHECK        = $C0000092;
  STATUS_FLOAT_UNDERFLOW          = $C0000093;
  STATUS_INTEGER_DIVIDE_BY_ZERO   = $C0000094;
  STATUS_INTEGER_OVERFLOW         = $C0000095;
  STATUS_PRIVILEGED_INSTRUCTION   = $C0000096;
  STATUS_STACK_OVERFLOW           = $C00000FD;
  STATUS_CONTROL_C_EXIT           = $C000013A;
var
  ErrCode: Byte;
begin
  case P.ExceptionCode of
    STATUS_INTEGER_DIVIDE_BY_ZERO:  ErrCode := 200; { reDivByZero }
    STATUS_ARRAY_BOUNDS_EXCEEDED:   ErrCode := 201; { reRangeError }
    STATUS_FLOAT_OVERFLOW:          ErrCode := 205; { reOverflow }
    STATUS_FLOAT_INEXACT_RESULT,
    STATUS_FLOAT_INVALID_OPERATION,
    STATUS_FLOAT_STACK_CHECK:       ErrCode := 207; { reInvalidOp }
    STATUS_FLOAT_DIVIDE_BY_ZERO:    ErrCode := 200; { reZeroDivide }
    STATUS_INTEGER_OVERFLOW:        ErrCode := 215; { reIntOverflow}
    STATUS_FLOAT_UNDERFLOW,
    STATUS_FLOAT_DENORMAL_OPERAND:  ErrCode := 206; { reUnderflow }
    STATUS_ACCESS_VIOLATION:        ErrCode := 216; { reAccessViolation }
    STATUS_PRIVILEGED_INSTRUCTION:  ErrCode := 218; { rePrivInstruction }
    STATUS_CONTROL_C_EXIT:          ErrCode := 217; { reControlBreak }
    STATUS_STACK_OVERFLOW:          ErrCode := 202; { reStackOverflow }
  else                              ErrCode := 255;
  end;
  RunErrorAt(ErrCode, P.ExceptionAddress);
end;

Posted in Algorithms, Development, Floating point handling, Software Development, Uncategorized | Leave a Comment »

floating point – SQL Server: Calculation with numeric literals requires to cast to obtain the right precision (via: Stack Overflow)

Posted by jpluimers on 2013/12/24

This has bitten me so many times, so I’m glad I found the below question/answers on StackOverflow.

When you perform calculations in SQL Server involving numeric literals, you have to take into account which precision you want your result to be, and CAST/CONVERT  the literals accordingly.

The reason is condensed to this statement by Lieven Keersmaekers:

SQL Server uses the smallest possible datatype.

He follows with examples to view the actual representation of a literal/expression using SQL_VARIANT_PROPERTY (which has been there since at least SQL Server 2000).

SELECT SQL_VARIANT_PROPERTY(1.0, 'BaseType')
SELECT SQL_VARIANT_PROPERTY(1.0, 'Precision')
SELECT SQL_VARIANT_PROPERTY(1.0, 'Scale')
SELECT SQL_VARIANT_PROPERTY(1.0, 'TotalBytes')

Read the rest of this entry »

Posted in Algorithms, Database Development, Development, Floating point handling, Software Development, SQL Server, SQL Server 2000, SQL Server 2005, SQL Server 2008, SQL Server 2008 R2, SQL Server 2012, SQL Server 7 | Leave a Comment »

SQL Server: some links on BULK IMPORT format files

Posted by jpluimers on 2013/12/04

From my link archive:

Note that for importing decimal/numeric columns, you have two options:

  1. Cast through FLOAT using a FORMAT file
  2. Use OpenRowSet with VARCHAR, then CAST afterwards
    Weird rounding for decimal while doing a bulk insert from a CSV.

Some more links on this:

–jeroen

Posted in Algorithms, CSV, Database Development, Development, Floating point handling, Software Development, SQL Server, SQL Server 2005, SQL Server 2008, SQL Server 2008 R2, SQL Server 2012 | Leave a Comment »

Delphi – Direct3D and the wrong FPU state: Now() function returns a wrong value (via: StackOverflow)

Posted by jpluimers on 2013/10/31

The question datetime – Delphi Now() function returns a wrong value – Stack Overflow is similar to my article Delphi – Michael Justin had strange floating point results when his 8087 FPU Control Word got hosed.

Good that stackoverflow user Anton Zhuchkov found out the cause himself: his answer indicates the Precision Control (and rounding) part of the FPU state got hosed by wrongly initializing the Direct3D device.

I edited his answer with some extra links to documentation.

Finally I’ve found the solution. I needed to specify the D3DCREATE_FPU_PRESERVE flag when creating a D3D device by D3D.CreateDevice.

Otherwise, without that flag, all floating point operations are performed with single precision. As the TDateTime is a simple Double, and Now() functions is consist of simple addition of date value to time value, it all get messed up by DirectX “smart” override.

Problem solved. It was a tricky one indeed. :)

–jeroen

via: datetime – Delphi Now() function returns a wrong value – Stack Overflow.

Posted in 8087, Algorithms, Delphi, Delphi 2005, Delphi 2006, Delphi 2007, Delphi 2009, Delphi 2010, Delphi 5, Delphi 6, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Development, Floating point handling, Software Development | 2 Comments »