Monday, 15 July 2019

How to monitor DotNetCore App in ElasticSearch using APM?

Steps:
  1. Download & Install ElasticSearch
  2. Download and Install Kibana
  3. Download and install Apm-Server
  4. Add Apm-Agent in to DotnetCore App
  5. Monitor your app in Kibana under Apm Tab.

1. Download & Install ElasticSearch

Download latest ElasticSearch MSI from here
https://www.elastic.co/downloads/elasticsearch
Install it with default settings and verify service up on http://localhost:9200/


2. Download and Install Kibana

Download latest Kibana Zip from https://www.elastic.co/downloads/kibana
Extract Zip by using 7zip if you facing any issue while extracting, in folder C:\Program Files\ElasticSearch\
Run Kibana as service:

Open CMD as administrator
sc create "ElasticSearch Kibana 7.2.0" binPath= "C:\Program Files\ElasticSearch\kibana-7.2.0-windows-x86_64\bin\kibana.bat" depend= "Elasticsearch" start= auto

Verify service up on http://localhost:5601/



Remove Kibana from service():
Open CMD as administrator
sc Delete "ElasticSearch Kibana 7.2.0"


3. Download and install Apm-Server

Now in Kibana go to APM http://localhost:5601/app/apm#/services?_g=()
Follow Windows setup instructions:
Download APM from https://www.elastic.co/downloads/apm

Follow install instruction by using Powershell.

Verify service up on http://localhost:8200/


4. Add Apm-Agent into DotnetCore App

Now follow application setup instruction:
For Dotnet core install following Nuget package with a pre-release check as per your requirement
• Elastic.Apm
• Elastic.Apm.All
• Elastic.Apm.AspNetCore
• Elastic.Apm.EntityFrameworkCore

And Add middleware UseElasticApm in starup.cs

public class Startup
{
  public void Configure(IApplicationBuilder app, IHostingEnvironment env)
  {
    app.UseElasticApm(Configuration);
    //…rest of the method
  }
  //…rest of the class
}


5. Monitor your app in Kibana under Apm Tab.

And verify tracing under APM







Thursday, 4 July 2019

Basic Docker Commands


Docker Image Pull/Run/remove command

Docker pull microsoft/mssql-server-linux:2017-latest

Docker run --name demosqlserver \
-p 1431:1433 \
-e "ACCEPT_EULA=Y" \
-e "SA_PASSWORD=password1234" \
-d microsoft/mssql-server-linux:2017-latest

Docker rm - f demosqlserver

Process a docker-compose.yaml command

Build and Run container
docker-compose up --build

Run container in background
docker-compose up -d

Above command run all services in background that could be verified by
Docker-compose ps


Building block
Docker Pull
Described above

Docker Build
Docker build . -t rajkrs/myappimage:latest

Docker Login
To login into docker hub account

Docker Push
Docker push rajkrs/myappimage:latest


Common commands:

docker container ls
List containers 
docker container rm ""
Remove one or more containers
docker rmi 
Removes one or more images. docker rmi my_image
docker container kill
Kill one or more running containers
docker ps
 Lists running containers
docker ps -a
List of all existing containers.
docker stop <imageid>
Stop specific image
docker kill $(docker ps -q)
kill all running containers
docker rm $(docker ps -a -q)
delete all stopped containers
Docker rm <containerid>
Remove a container from registry
docker rmi $(docker images -q)
delete all images
Docker start/stop <containerid>
Run a container







Wednesday, 3 July 2019

Open source Security Scan | Code analysis | Security suggestion | Open Source Security Guard Tools for .net, dotnetcore.


Find OWASP official open source tools list here.


As a dot.net programmer we can have look on following two which include security audits for .net, dotnetcore applications expose and suggest SQL injections, LDAP injections, CSRF/XSS, Cryptography weakness with Visual Studio Extension   :


How to use such tools?
Steps:
  1. Install Extension Puma Scan.
  2. Enable Full solution Analysis in C# text editor option
  3. Build your project and found info.





Thursday, 20 June 2019

VS Code Create Multiple DotnetCore App and Debug

Just mentioning few simple steps and command line by following which we can start working with Visual Studio Code. 

Open terminal for specific created folder.

Create a new solution file, same name as folder
dotnet new sln

Create a new folder for project
mkdir app.webapi

Go to project folder
cd app.webapi

Create a new dot net core web api project:
Dotnet new webapi

Back to root folder:
Cd ..

Create a new folder for 2nd project
Mkdir app.web

Go to folder:
Cd app.web

Create csproj for other dotnet core web project:
Dotnet new web

Back to root folder:
Cd ..

Add webapi project to solution:
dotnet sln VsCodeMultiAppLaunch.sln add .\app.webapi\app.webapi.csproj

Add web project to solution:
dotnet sln VsCodeMultiAppLaunch.sln add .\app.web\app.web.csproj

Back to root folder:
Cd ..

Create new folder for web api data access project:
Mkdir app.webapi.dal

Go to newly created folder:
Cd app.webapi.dal

Create a new class library:
Dotnet new classlib

Back to root folder:
Cd ..

Add app.webapi.dal project to solution:
dotnet sln VsCodeMultiAppLaunch.sln add .\app.web\app.web.csproj

After execution of above command we can verify content under workstation.



Add app.webapi.dal refrence to app.webapi project.
dotnet add .\app.webapi\app.webapi.csproj reference .\app.webapi.dal\app.webapi.dal.csproj

Or else add in app.webapi\app.webapi.csproj manually:
<ItemGroup>
<ProjectReference Include="..\app.webapi.dal\app.webapi.dal.csproj" />
</ItemGroup>

Build all project:
Dotnet build

Debug
Press F5
It will create a new launch.json under .vscode


Now press 5 to Debug app.webapi project.

If you want to debug multiple project together, copy and paste configuration and modify accordingly.

{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": ".NET Core Launch (web api)",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
"program": "${workspaceFolder}/app.webapi/bin/Debug/netcoreapp3.0/app.webapi.dll",
"args": [],
"cwd": "${workspaceFolder}/app.webapi",
"stopAtEntry": false,
"serverReadyAction": {
"action": "openExternally",
"pattern": "^\\s*Now listening on:\\s+(https?://\\S+)"
},
"env": {
"ASPNETCORE_ENVIRONMENT": "Development",
"ASPNETCORE_URLS": "https://localhost:5000"
},
},
{
"name": ".NET Core Launch (web)",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
"program": "${workspaceFolder}/app.web/bin/Debug/netcoreapp3.0/app.web.dll",
"args": [],
"cwd": "${workspaceFolder}/app.web",
"stopAtEntry": false,
"serverReadyAction": {
"action": "openExternally",
"pattern": "^\\s*Now listening on:\\s+(https?://\\S+)"
},
"env": {
"ASPNETCORE_ENVIRONMENT": "Development",
"ASPNETCORE_URLS": "https://localhost:5001"
},
"sourceFileMap": {
"/Views": "${workspaceFolder}/Views"
}
},
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickProcess}"
}
]
}

Now you will be getting

You can click one by one on Launch and will able to debug each project.

Wednesday, 12 June 2019

Create a simple Retry Policy by using Polly in any fault.

Polly a .NET fault-handling wrapper that allows developers to express policies in thread safe manner.
Policies could be made over:

  • Retry
  • Circuit Breaker
  • Timeout
  • Bulkhead Isolation
  • Fallback.

More: <https://github.com/App-vNext/Polly

Simple Demo:

using Polly;
using System;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
 
namespace PollyRetry
{
 
    class Program
    {
        static void Main(string[] args)
        {
            HttpRequestDemo();
            Console.WriteLine("Done!");
            Console.ReadLine();
        }
 
        private static void HttpRequestDemo()
        {
 
 
            HttpStatusCode[] httpStatusCodesWorthRetrying = {
                HttpStatusCode.RequestTimeout, // 408
                HttpStatusCode.InternalServerError, // 500
                HttpStatusCode.BadGateway, // 502
                HttpStatusCode.ServiceUnavailable, // 503
                HttpStatusCode.GatewayTimeout, // 504
                HttpStatusCode.NotFound // 404
            };
 
            var httpRetryPolicy = Policy.HandleResult<HttpResponseMessage>(r => httpStatusCodesWorthRetrying.Contains(r.StatusCode))
            .OrResult(r => r.StatusCode == HttpStatusCode.BadGateway)
            .WaitAndRetry(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)), (extime) =>
            {
                Console.WriteLine($"Unable to complete operation after {time.TotalSeconds:n1}s ({ex.Result.StatusCode})");
            });
 
            using (var client = new HttpClient())
            {
                client.BaseAddress = new Uri("http://localhost:51976/");
                client.DefaultRequestHeaders.Accept.Clear();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                var response = httpRetryPolicy.ExecuteAndCapture(() =>
                {
                    return client.GetAsync("api/order").Result;
                }).FinalHandledResult;
 
                if (response != null && response.IsSuccessStatusCode)
                {
                    Console.WriteLine("Got valid response from API");
                }
            }
 
 
        }
    }
}


We can also create multiple policy to handle multiple faults, like server is down and we are unable to establish connection with the service. In that case at initial call above code will produce exception. So, I made some changes in code, created two policies and wrapped first one with 2nd once.

using Polly;
using System;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
 
namespace PollyRetry
{
 
    class Program
    {
        static async System.Threading.Tasks.Task Main(string[] args)
        {
            await HttpRequestDemoAsync();
            Console.WriteLine("Done!");
            Console.ReadLine();
        }
 
        private static async System.Threading.Tasks.Task HttpRequestDemoAsync()
        {
 
            int retryCount = 5;
 
            HttpStatusCode[] httpStatusCodesWorthRetrying = {
                HttpStatusCode.RequestTimeout, // 408
                HttpStatusCode.InternalServerError, // 500
                HttpStatusCode.BadGateway, // 502
                HttpStatusCode.ServiceUnavailable, // 503
                HttpStatusCode.GatewayTimeout, // 504
                HttpStatusCode.NotFound // 404
            };
 
 
            var httpRetryPolicy = Policy.HandleResult<HttpResponseMessage>(r => httpStatusCodesWorthRetrying.Contains(r.StatusCode))
            .OrResult(r => r.StatusCode == HttpStatusCode.BadGateway)
            .WaitAndRetryAsync(retryCountretryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)), (exceptioncalculatedWaitDuration) =>
            {
                Console.WriteLine($"Unable to complete operation after {calculatedWaitDuration.TotalSeconds:n1}s ({exception.Result.StatusCode})");
            });
 
            var exceptionPolicy = Policy
            .Handle<Exception>()
            .WaitAndRetryAsync(retryCountretryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)), (exceptioncalculatedWaitDuration) =>
            {
                Console.WriteLine($"Error while performing operation after {calculatedWaitDuration.TotalSeconds:n1}{exception.Message}");
            })
            .WrapAsync(httpRetryPolicy);
 
 
            using (var client = new HttpClient())
            {
                client.BaseAddress = new Uri("http://localhost:51976/");
                client.DefaultRequestHeaders.Accept.Clear();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                var response = (await exceptionPolicy.ExecuteAndCaptureAsync(() =>
                {
                    return client.GetAsync("api/order");
                })).FinalHandledResult;
 
                if (response != null && response.IsSuccessStatusCode)
                {
                    Console.WriteLine("Got valid response from API");
                }
            }
        }
    }
}

Now you can see variation in fault message

In case of DotNetCore we can use Microsoft.Extensions.Http.Polly.

Thursday, 30 May 2019

Distributed Microservice Architecture with Rabbitmq and DotNetCore

When we say distributed, suddenly 3rd parties message broker comes in our mind like Azure Service Bus or RabbitMQ. Let explore RabbitMQ and its implementation in DotnetCore.

RabbitMQ is a message broker: it accepts and forwards messages. It use AMQP Advanced Message Queuing Protocol.

Key Points:

  • Message: a package for information, usually composed of two parts; headers, containing metadata, and a body, containing a binary package containing the actual message itself
  • Producer: Whoever creates and sends a message.
  • Consumer: Whoever receives and reads a message
  • Queue: A communication channel that enqueues messages for later retrieval by one or more consumers.
  • Exchange: A queue aggregator that routes messages to queues based on some predefined logic


Run RabbitMq through docker:
docker run -d --hostname my-rabbit --name some-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management

Then go to http://localhost:15672 and use following credentials.
Username:guest
Password:guest

Advantage of using it?
Checkout https://www.rabbitmq.com/features.html

How it can fit in to distributed micro service architecture?
Micro soft itself described in one of his GitHub implementation. With this design.
https://github.com/dotnet-architecture/eShopOnContainers

Any simple running demo of above design by using RabbitMq.Client in DotNetCore ?
Yes, https://github.com/rajkrs/DotnetCoreWebApiAndRabbitMq

I don’t want to use RabbitMq Dotnet Client, any supplements?
EasyNetQ is more popular than other, check it out http://easynetq.com/

Any simple running demo of above design by using EasyNetQ?
Yes, https://github.com/rajkrs/EasyNetQDotNetCore

Tuesday, 23 April 2019

Custom Model Binding through header in DotNetCore

Now the day with Asp.net core we have facility to bind object through exact binding by using attributes [FromHeader], [FromQuery], [FromRoute], [FromForm], [FromBody] and [FromServices] etc.

In this post we are focusing on FromHeader in which we have one constrain that it doesn't not resolve Complex Type.
But we can overcome from this constrain by using ModelBinder by using which we can customize Binder in asp.net core.

Scenario
Let's suppose that we have a Complex Model

public class UserLogin
{
    public int UserId { getset; }
    public string Password { getset; }
    public DateTime? RequestTime { getset; }
}

Now we want to make a get call where we are putting JSON value from Header with key RequestModel.

Now what we are expecting that, from RequestModel header key Model should resolve there in controller where we are using FromHeader attribute.
But unfortunately it is null.

To resolve this problem we can have 2 quick solution.
1. Create Custom Binder that implement IModelBinder, and use additional hint everywhere, when binding required from header.
2. Create a Custom Binder from IModelBinder and Custom provider from IModelBinderProvider, point Binder to Provider and Use provider in ConfigureServices globally so without adding additional hint model will be resolve.  
Note: In this example we just resolve only one header key RequestModel.

Solution 1:
Create a simple HeaderComplexModelBinder.cs
using Microsoft.AspNetCore.Mvc.ModelBinding;
using Newtonsoft.Json;
using R6.Core.Constants;
using System;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Threading.Tasks;
 
namespace R6.Service.Filter
{
    [ExcludeFromCodeCoverageAttribute]
    public class R6ModelBinder : IModelBinder
    {
        public Task BindModelAsync(ModelBindingContext bindingContext)
        {
            if (bindingContext == null)
            {
                throw new ArgumentNullException(nameof(bindingContext));
            }
 
            if (bindingContext == null)
            {
                throw new ArgumentNullException(nameof(bindingContext));
            }
 
            var headerKey = bindingContext.ModelMetadata.BinderModelName;//Resolve Name from FormHeader(Name = "Key")
            if (string.IsNullOrEmpty(headerKey))
            {
                headerKey = Constant.CUSTOM_HEADER_NAME;
            }
            var headerValue = bindingContext.HttpContext.Request.Headers[headerKey].FirstOrDefault();
            var modelType = bindingContext.ModelMetadata.ModelType;
 
            if (!string.IsNullOrEmpty(headerValue))
            {
                bindingContext.Model = JsonConvert.DeserializeObject(headerValuemodelType);
                bindingContext.Result = ModelBindingResult.Success(bindingContext.Model);
            }
            return Task.CompletedTask;
        }
    }
}

And change action like this, and you found resolved Model.
[ModelBinder(typeof(HeaderComplexModelBinder))] UserLogin userLogin
Solution 2:
We need to keep HeaderComplexModelBinder.cs and create an additional HeaderComplexModelBinderProvider.cs

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ModelBinding;
using Microsoft.AspNetCore.Mvc.ModelBinding.Binders;
using System;
using System.Linq;
 
namespace CoreApp.WebApi
{
    public class HeaderComplexModelBinderProvider : IModelBinderProvider
    {
 
        public IModelBinder GetBinder(ModelBinderProviderContext context)
        {
            if (context == null)
            {
                throw new ArgumentNullException(nameof(context));
            }
 
            if (context.Metadata.IsComplexType)
            {
                var x = context.Metadata as Microsoft.AspNetCore.Mvc.ModelBinding.Metadata.DefaultModelMetadata;
                var headerAttributex.Attributes.Attributes.Where(a => a.GetType() == typeof(FromHeaderAttribute)).FirstOrDefault();
                if (headerAttribute != null)
                {
                    return new BinderTypeModelBinder(typeof(HeaderComplexModelBinder));
                }
                else
                {
                    return null;
                }
            }
            else
            {
                return null;
            }
        }
    }
}

And add provider inside MVC service as option in ConfigureServices.
services.AddMvcCore(options =>
{
options.ModelBinderProviders.Insert(0, new HeaderComplexModelBinderProvider());
}

Now make your action very simple like this and you found resolved Model. :-)