[WIP] Compact Filter protocol messages

[MsBarber]

Hello, this is a work in progress for light client protocol messages. Still a WIP because of lack of proper tests. I would like to give credit to @lontivero for opening #401, getting the ball rolling on the issue, and giving me a solid place to start.

There has been a few changes regarding filter types, for now the only supported filter is "Basic". Therefore I have commented out the conditional checks for "Extended" filter type, also commented out the enum value.

For the payloads I have included all the fields included in bip157. I included the "length" fields which are technically not necessary to save in the object, as far as I am concerned.

I have overrode ToString() for testing purposes.

I appreciate all comments and feed back given.

The end goal is to eventually create a compact filter light client, similar to neutrino on top of Nbitcoin.

[nopara73]

Btw, are there already any serving full nodes on the mainnet?

[MsBarber]

The only node on mainnet that I am aware of is the BTCD node that I myself am running.

[nopara73]

@MsBarber Is it merged to btcd already? Last time I checked it wasn't.

In fact I'm looking at Roasbeef's btcd fork: https://github.com/Roasbeef/btcd is "This branch is 137 commits ahead btcsuite:master." Of course this may be something else.

[MsBarber]

@nopara73 As far as I am concerned development has moved over to https://github.com/btcsuite/btcd
Also this implementation of the reference client does support compact (commited) filters, https://github.com/btcsuite/btcd/blob/2a560b2036bee5e3679ec2133eb6520b2f195213/config.go#L156 you can see the option to disable it here.

I have a node running and have been sending cf related messages and getting appropriate responses.

If you would like to connect to my node and give it a spin I would not mind sharing the IP, its mainnet.

The BTCD node does take a good amount of time to sync if you chose to spin it up.

[MsBarber]

Unsure as to why AppVeyor is having hard time locating the FilterType?

[NicolasDorier]

There is also several warning to fix.

[NicolasDorier]

Can you rebase and fix the build?

[MsBarber]

@NicolasDorier Hey Nick, Appveyor seems to still be failing, however travis is passing.
The error from appveyor makes no sense as the filter type is clearly being found by travis.

[lontivero]

We use to say: the SQL is not broken. Take a closer look at the console log:

NBitcoin -> C:\projects\nbitcoin\NBitcoin\bin\Release\net452\NBitcoin.dll
  NBitcoin -> C:\projects\nbitcoin\NBitcoin\bin\Release\net461\NBitcoin.dll
Protocol\Payloads\CompactFilterCheckPointPayload.cs(53,10): error CS0246: The type or namespace name 'FilterType' could not be found (are you missing a using directive or an assembly reference?) [C:\projects\nbitcoin\NBitcoin\NBitcoin.csproj]
Protocol\Payloads\CompactFilterCheckPointPayload.cs(21,41): error CS0246: The type or namespace name 'FilterType' could not be found (are you missing a using directive or an assembly reference?) [C:\projects\nbitcoin\NBitcoin\NBitcoin.csproj]
  NBitcoin -> C:\projects\nbitcoin\NBitcoin\bin\Release\netstandard1.3\NBitcoin.dll
  NBitcoin -> C:\projects\nbitcoin\NBitcoin\bin\Release\netcoreapp2.1\NBitcoin.dll
  NBitcoin -> C:\projects\nbitcoin\NBitcoin\bin\Release\netstandard2.0\NBitcoin.dll

It compiles okay for all the targets except for net461. That's why it pass green in travis. Next, remove all the targets except net461 and build again, it has to fail.

[lontivero]

Just to make sure I cloned the repo following the instructions shown in the console log:

git clone -q --depth=1 https://github.com/MetacoSA/NBitcoin.git c:\projects\nbitcoin
git fetch -q origin +refs/pull/548/merge:
git checkout -qf FETCH_HEAD

Then built the project with:

dotnet build

I expected to fail because I work on Linux and don't have .NET Framework. It failed and gave me the same error:

/usr/share/dotnet/sdk/2.1.500/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.6.1" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lontivero/GitHub/nb/NBitcoin.Tests/NBitcoin.Tests.csproj]
/usr/share/dotnet/sdk/2.1.500/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.6.1" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lontivero/GitHub/nb/NBitcoin.Altcoins/NBitcoin.Altcoins.csproj]
/usr/share/dotnet/sdk/2.1.500/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.6.1" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lontivero/GitHub/nb/NBitcoin/NBitcoin.csproj]
/usr/share/dotnet/sdk/2.1.500/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5.2" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lontivero/GitHub/nb/NBitcoin.TestFramework/NBitcoin.TestFramework.csproj]
/usr/share/dotnet/sdk/2.1.500/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5.2" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lontivero/GitHub/nb/NBitcoin/NBitcoin.csproj]
Protocol/Payloads/CompactFilterCheckPointPayload.cs(53,10): error CS0246: The type or namespace name 'FilterType' could not be found (are you missing a using directive or an assembly reference?) [/home/lontivero/GitHub/nb/NBitcoin/NBitcoin.csproj]
Protocol/Payloads/CompactFilterCheckPointPayload.cs(21,41): error CS0246: The type or namespace name 'FilterType' could not be found (are you missing a using directive or an assembly reference?) [/home/lontivero/GitHub/nb/NBitcoin/NBitcoin.csproj]
/usr/share/dotnet/sdk/2.1.500/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5.2" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lontivero/GitHub/nb/NBitcoin.Altcoins/NBitcoin.Altcoins.csproj]
    0 Warning
    8 Error(s)

[lontivero]

Ohhh it is simply that some files has the #if !NOSOCKET and others don't!!! so, yes, the FilterType was not defined.

[MsBarber]

@lontivero

Ohhh it is simply that some files has the #if !NOSOCKET and others don't!!! so, yes, the FilterType was not defined.

Lol, yes I also noticed this not too long ago and have applied the fix. Thank you so much for the help!

Hmmm, is it necessary to have this wrapped in '#if !NOSOCKET'? I saw a mention of in #110 and it seems it was just to take care of some non portable IpEndPoint code.

I am going to remove them and see if it builds, if so there is no need for them I think.

[NicolasDorier]

Can you rebase?

[MsBarber]

TravisCI was unable to located dotnet-sdk-2.1?

E: Unable to locate package dotnet-sdk-2.1 E: Couldn't find any package by glob 'dotnet-sdk-2.1' E: Couldn't find any package by regex 'dotnet-sdk-2.1' The command "sudo apt-get install -qq dotnet-sdk-2.1" failed and exited with 100 during .

[NicolasDorier]

Retrying

[NicolasDorier]

Can you add some tests? Is there some test vector in the BIP so we can try serializing and deserializing?